Detecting of business email compromise

ABSTRACT

A system and method for detection of email risk automatically is disclosed.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/917,197, filed Jun. 30, 2020, which is a U.S. patent application Ser.No. 15/414,489, filed Jan. 24, 2017, which issued on Jul. 21, 2020 asU.S. Pat. No. 10,721,195, which claims priority to U.S. ProvisionalApplication No. 62/287,378, filed Jan. 26, 2016, each of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

Business Email Compromise (BEC) is a type of scam that has increaseddramatically in commonality in the recent past. In January 2015, the FBIreleased stats showing that between Oct. 1, 2013 and Dec. 1, 2014, some1,198 companies reported having lost a total of $179 million in BECscams, also known as “CEO fraud.” It is likely that many companies donot report being victimized, and that the actual numbers are muchhigher. There therefore exists an ongoing need to protect users againstsuch scams.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a system fordetecting scam.

FIG. 2A is a flow diagram illustrating an embodiment of a process fordetecting scam.

FIG. 2B is a flow diagram illustrating an embodiment of a process fordetecting scam.

FIG. 2C is a flow diagram illustrating an embodiment of a process fordetecting scam.

FIG. 2D is a flow diagram illustrating an embodiment of a process fordetecting scam.

FIG. 3 illustrates an example process to determine that an account is afriend.

FIG. 4 illustrates an example process to determine that an email senderis trusted.

FIG. 5 illustrates an embodiment of a simplified non-monotonicallyincreasing filter.

FIG. 6 illustrates an alternative embodiment of a non-monotoniccombining logic.

FIG. 7 illustrates a second alternative embodiment of a non-monotoniccombining logic.

FIG. 8 illustrates an example process for classification of primaryrisks associated with an email, using a non-monotonically increasingcombining component.

FIG. 9 illustrates an example embodiment of a process to identify whatmessages should be quarantined based on both high risk and a reasonablelikelihood of being legitimate.

FIG. 10 illustrates an embodiment of a quarantine process using asecondary channel for release of quarantined messages.

FIG. 11 illustrates an example embodiment of a process for processing ofa quarantined email message.

FIG. 12 illustrates an example of the three stages in one embodiment ofa 2FA confirmation process.

FIG. 13 illustrates an example embodiment of processing associated withsending a request to an account associated with the apparent sender ofan email.

FIG. 14 illustrates an example embodiment of a request.

FIG. 15 illustrates an example embodiment of a request.

FIG. 16 illustrates an example embodiment of a cousin clearinghouse.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

A BEC scam usually begins with the thieves either phishing an executiveand gaining access to that individual's inbox, or emailing employeesfrom a lookalike domain name that is, for example, one or two lettersoff from the target company's true domain name. For example, if thetarget company's domain was “example.com” the thieves might register“example.com” (substituting the letter “L” with the numeral 1) or“example.co,” and send messages from that domain. Other times, thethieves will spoof an email, e.g., using a mail server setup to act asan open relay, which permits them to send bogus emails with a realdomain name that is not theirs. Yet other times, the thieves may createa personal email account with a user name suggesting that the emailaccount belongs to the CEO, and then email the CEO's secretary with arequest. Commonly, the thieves request that the recipient transfer moneyfor some business transaction. In many cases, the thieves have studiedthe targeted organization well enough to know what kind of request willseem reasonable, making them likely to be more successful. For example,a thief can gain access to an internal email account, like the CEO's,and find a previous legitimate invoice that is then modified to become ascam.

Other, technically similar scams also face consumers. One example ofthis is the so-called “stranded traveler scam”, which typically involvesa friend of the victim who was robbed in a foreign country and needs aquick loan to get home. Other related scams include scams where youngadults supposedly are jailed in a foreign country, and need help fromgrandparents. Many times, scams like these use accounts that have beencompromised, e.g., in phishing attacks. Sometimes, spoofing is used, orother methods of deceit, including registration of email accounts withnames related to the person in supposed need. What is common for all ofthese scams is that they use deception, and commonly take advantage ofpre-existing trust relationships between the intended victim and theparty in supposed need.

When BEC scams are referred to in this document, they refer to thecollection of scams that have the general format of the BEC scam, whichincludes but is not limited to stranded traveler scams, imprisoned inMexico scams, phishing emails, and other emails that suggestfamiliarity, authority, friendship or other relationship. Many targetedscams fall in this category, and scams of these types can be addressedby using the techniques described herein.

Unlike traditional phishing scams, spoofed emails used in CEO fraudschemes and related scams, such as those described above, are unlikelyto set off traditional spam filters, because these are targeted phishingscams that are not mass emailed, and common spam filters rely heavily onthe quantity of email of a certain type being sent. Also, the crooksbehind them take the time to understand the target organization'srelationships, activities, interests and travel and/or purchasing plans.This makes the scam emails look rather realistic—both to theirrecipients and to traditional spam filters.

Traditional spam filtering is designed to detect typical spam. This istypically sent in high volume, has low open rates, and even lowerresponse rates. It is commonly placed in the spam folder by therecipient (if not already done so by the spam filter). It commonlycontains a small set of keywords, corresponding to the products that aremost profitable for spammers to sell. These keywords are typically notused in non-spam email traffic. To avoid detection by spam filters,spammers commonly obfuscate messages, e.g., write V-!-@-G.R-A instead of“Viagra”. This commonly helps the spammers circumvent spam filters, butthe message is typically still clear to the recipient.

In contrast, a typical BEC scam message is sent to only a small numberof targeted recipients, such as one or two recipients within anorganization. If similar messages are sent to recipients in otherorganizations, those are typically not verbatim copies, as there is afair amount of customization, much of which is guided by contextualinformation obtained from data breaches, compromised accounts, andpublicly available information, including social networks. There aretypically no keywords specific to BEC emails—instead, BEC scammersattempt to mimic the typical emails of the people they interact with. Asa result, there is typically no need for obfuscation. BEC scammers maypurchase or register new domain names, like example.com above, solelyfor the purpose of deceiving users within one specific organizationtargeted by the scammer, and may spend a significant amount of effortcustomizing their emails to make them credible, based on contextualinformation related to the intended victims. These factors contribute tomake traditional/existing spam filters fail to detect BEC scam emails.

In some embodiments, the techniques described herein address theproblems of email scams, such as BEC scams, using a set of detectioncomponents. While example embodiments involving email are describedbelow, the techniques described herein can variously be adapted toaccommodate any type of communication channel, such as chat, (e.g.,instant messaging (IM)), text (e.g., short message service (SMS)), etc.,as applicable.

In various embodiments, the detection components include, but are notlimited to, components to detect deceptive email content; to detectdeceptive domains; to detect deceptive email addresses; to detect emailheader structures associated with deceptive practices; to detectdeceptive attachments; and to detect hyperlinked material that isassociated with deceptive emails.

Furthermore, in some embodiments, the outputs of at least two deceptiondetection components are combined in a way that limits error rates, forexample, using a non-monotonic combining logic that triggers oncombinations of the above described deception detection components.Further details regarding this logic will be described below. In someembodiments, the logic reduces error rates by mirroring scammerstrategies and associated uses of approaches that cause the deceptiondetection components to trigger. In some embodiments, this reduces falsenegatives. At the same time, in some embodiments, the logic reducesfalse positives by not blocking benevolent emails, even if these causethe triggering of deception detection components, for example, as longas these are not triggered according to patterns indicative of commonscammer strategies.

As will be illustrated in further detail below, the techniques describedherein mitigate the threat associated with Business Email Compromise andassociated scams. In some embodiments, this is done by detectingstructural persuasion attempts. In some embodiments, this is in contrastto verbal persuasion attempts, which include text-based appeals in thecontent portion of a message. In some embodiments, structural persuasionrelates to use of deceptive header information intended to cause therecipient of an email to be inclined to accept a message as legitimateand safe.

In some embodiments, the use of second factor authentication (2FA) forconfirmation is beneficial to avoid risk. For example, if Alice sends anemail to her broker, Bob, asking Bob to sell some of her stock, then itcan be beneficial for Bob to confirm with Alice before performing thesale. This avoids performing transactions as a result of attacks, suchas a spoofing attack in which Eve is sending a spoofed message to Bob,appearing to come from Alice. It also mitigates the threat associatedwith malware and stolen computers. For example, consider a setting whereEve places malware on Alice's computer, causing an email to be sent fromAlice to Bob, in which Bob is asked to sell some of Alice's stock. Inthese examples, using a 2FA for confirmation reduces the threat, as ifEve does not have the ability to receive the 2FA request and respond toit on Alice's behalf, then the email request will be ignored by Bob. Insome embodiments, the 2FA confirmation requests include SMS messages ormanually placed phone calls. Existing systems for sending 2FAconfirmation requests are not automated. Instead, Bob reads his emailfrom Alice, and determines in a case-by-case basis whether to initiate a2FA confirmation request. Occasionally, Bob may make a mistake or behurried by a high-priority request, thereby deciding to ignore the 2FAconfirmation. Scammers may trick Bob to omit the request. In someembodiments, the techniques described herein automate the determinationof when to send a 2FA confirmation request, and integrate theconfirmation with the delivery of the email. This way, Bob will notreceive the email from Alice until Alice has confirmed it, unless it isan email that does not require a confirmation, in which case it will bedelivered immediately.

Traditional spam filters typically have a logic that is monotonicallyincreasing. What this means is that they may have combining logicfunctions that generate a filtering decision from two or more detectioncomponents, such as one velocity detector and one reputation detector,and where a “higher” detection on either of these result in a higherprobability of blocking the email. For example, the output of thevelocity detector may be three levels, corresponding to low, medium, andhigh velocities. Similarly, the output of the reputation detector may bethree levels, corresponding to low, medium, and high reputation risk.The combining logic function may determine that a message is undesirableif it results in a high velocity level, a high reputation risk level, ora medium level if both the velocity detector and the reputationdetection components output medium levels. This traditional combininglogic is monotonically increasing, and works in a way that can bedescribed as “additive”: if any filter outputs a “higher” detectionscore, that means that it is more likely that the email will be blocked,as individual scores from different detection components are combined ina way in which each score contributes toward reaching a threshold in amanner that does not depend on the other scores. If the threshold isreached, a filter action is performed.

In contrast, in one embodiment, the disclosed scam detector (alsoreferred to herein as “the system”) corresponds to a logic combinationfunction that is not monotonically increasing. This type of function isreferred to herein as “non-monotonically increasing.” For example,suppose that a first and a second detector each have three possibleoutputs, which for illustrative purposes, are referred to as low,medium, and high. In some embodiments, the combining logic functiondetermines that an email is not desirable if the first detector outputshigh and the second detector outputs low; the first detector outputs lowand the second detector outputs high; or both generate a medium output;but otherwise determines that the email is desirable. In this example,it is clear that neither detector generates an output from which aclassification decision can be made without also taking the output ofthe other detector into consideration. It is also clear in this examplethat at least one of the detectors produces an output for which onevalue is not always indicative of a safe email, but sometimes that valueis indicative of an unsafe email. Seen another way, in some embodiments,the results of the individual detectors are combined using a combiningfunction whose operations depend on at least one of the scores and typesof the individual detectors. In some embodiments, such a detectoridentifies what other detectors are relevant for the classification, andhow to combine the scores and types from those.

While the above examples describe monotonically increasing andnon-monotonically increasing functions in the context of emailclassification, the techniques described herein can be applied to moredetectors than two, and to different types of detector outputs, such asbinary detector outputs and detector outputs with more than threepossible options. In some embodiments, the detector outputs are ofdifferent types for different detectors, such as a first detector with abinary output and a second detector with an output that can take tendifferent values. In some embodiments, the detector outputs can berepresented as numeric values, Boolean values, class memberships, or anyother appropriate types of values. Detectors can be implemented insoftware, hardware or a combination of these, and in some embodiments,may utilize some manual curation in cases where, for example, anautomated classification is not supported by the system rules for aparticular input email message.

The non-monotonic logic is described in further detail in the combininglogic section below, where example pseudocode is provided, illustratingan example embodiment of the techniques described herein. One exampleelement of relevance to the non-monotonic evaluation is theclassification of the sender being, or not being, a trusted party. Inone embodiment, a trusted sender is what is defined as a “friend” or an“internal” party in the example embodiment below. In another embodiment,a trusted sender is a party who the recipient has an entry for in his orher address book; is connected to on a network (e.g., social networksuch as Facebook or LinkedIn); has chatted or placed phone/video callsusing a communications application/program such as Skype or similarsoftware; or a combination of such properties. In one exampleembodiment, two associated parties share a list of trusted parties; ifone email sender is qualified as a trusted party for one of theassociated parties, then the same email sender is also automatically orconditionally qualified as a trusted party for the second associatedparty. Possible example conditions include the two associated partiesbeing members of the same organization; having configured theirrespective systems to allow for the exchange of information related towho is a trusted party; conditions relating to the certainty of theclassification and a minimum required certainty configuration of thesecond associated party; and any combination of such conditions. Furtherdetails regarding determining what users are trusted are describedbelow.

In some embodiments, the non-monotonic logic causes a differentevaluation of messages sent from trusted senders and non-trustedsenders. For example, in the example embodiment below, the presence ofan untrusted reply-to address is associated with risk when it is part ofa message from a trusted sender, but not from a non-trusted sender(e.g., from=bob@example.com is not the same as reply-to=bob@exampe.com).Similarly, in some embodiments, spoof indicators are associated withrisk in a message from a trusted sender, but not from a non-trustedsender. Conversely, in some embodiments, deceptive links, deceptiveattachments, deceptive domain names, deceptive email addresses, and thelike are associated with risk primarily in messages from non-trustedparties. In other words, in some embodiments, the risk evaluation logicdescribed herein is not “additive” in that the presence of an indicatorimplies greater risk in one context, while lesser risk in anothercontext. In some embodiments, the non-monotonic logic associated withthe risk evaluation maps to the business strategy of the scammers, wherethis business strategy corresponds to how they typically carry out theiracts of trying to scam recipients.

Described herein are also techniques for determining when an emailaddress is potentially deceptive. In some embodiments, a first componentof this determination determines the similarity of two or more emailaddresses, using, for example, string comparison techniques specificallydesigned to compare email addresses and their associated display nameswith each other. In some embodiments, this comparison is made withrespect to display name, user name, domain, TLD, and/or any combinationsof these, where two addresses can be compared with respect to at leastone such combination, which can include two or more. In someembodiments, this first component also includes techniques to matchconceptually similar strings to each other, where the two strings maynot be similar in traditional aspects. For example, the words “Bill” and“William” are not closely related in a traditional string-comparisonsense; however, they are conceptually related since people named“William” are often called “Bill”. Therefore, an email address with adisplay name “Bill” has a similar meaning to an email address with adisplay name “William”, even though the two are not similar in atraditional string comparison sense. Furthermore, the words “mom” and“rnorn” are not very similar in a traditional string comparison sense,since one is a three-letter word and the other a five-letter word, andthese two words only have one letter in common. However, they arevisually related since “m” looks similar to “rn”. This similarity may begreater for some fonts than for other, which is another aspect that isconsidered in one embodiment. In some embodiments, a string comparisontechnique that adds conceptual similarity detection to traditionalstring comparison improves the ability to detect deceptive emailaddresses. This can also include the use of unicode character sets tocreate homographs, which are characters that look like other characters,and which can be confused with those.

In some embodiments, a second component of the determination of whetheran email address is potentially deceptive relies on the context in whichthis is used. This is another example of a non-monotonic filterfunction. In some embodiments, if an email address of the sender of anemail corresponding to a non-trusted party is similar to that of atrusted party associated with the recipient of the email, then that isdeceptive, as the sender may attempt to mimic a trusted party. On theother hand, if the sender of an email is trusted, then having a reply-toaddress that is similar to the sender email address is deceptive. Forexample, a scammer can gain access to an account and send emails tofriends of the account owner but modifies the reply-to email to asimilarly looking address so that the real account holder does not seeresponses. Therefore, based on the trust relationship, the notion of“deceptive” changes meaning.

Another example of a non-monotonic aspect of the techniques disclosedherein is the presence of a reply-to address. In some embodiments, itmatters less whether a non-trusted sender has a reply-to address, andthis should not affect the filtering decision; on the other hand, itdoes matter whether a trusted sender has a reply-to address. If thisreply-to address is deceptive with respect to the sender address, thatis treated as a reason for taking a filtering action. In one embodiment,the fact that an email has a reply-to address—independently of whetherit is deceptive—where the reply-to address is not previously associatedwith the sender, is sufficient to flag the email if the sender is atrusted party. In various embodiments, flagged emails can be blocked,quarantined, marked up, or otherwise processed to reduce the riskassociated with them. The same is not true for a sender who is not atrusted party.

In one embodiment, the available filtering decisions are conditional forat least some of the detection components. For example, if it isdetermined that an email is sent from a non-trusted party, then it isacceptable to block it if it contains some elements associated with highrisk. If the apparent sender of the email is a trusted party and theemail headers contain a deceptive reply-to address, then it is alsoacceptable to block the message. If the apparent sender of the email isa trusted party and there is a new reply-to address that is notdeceptive, then it is not acceptable to block the email, but moreappropriate to quarantine, mark up, or otherwise flag the email.Similarly, if the apparent sender of the email is a trusted party andthere is no reply-to address but content associated with risk, thenbased on the level of risk, the message may either be marked up ortagged, or simply let through, if the risk is not very high. Instead ofblocking emails that are evaluated to be high-risk from a scamperspective as well as possibly having been sent by a trusted party, theemails can be marked up with a warning, sent along with a notificationor warning, quarantined until a step-up action has been performed, orany combination of these or related actions. One example step-up actioninvolves the filtering system or an associated system automaticallysending a notification to the apparent sender, asking for a confirmationthat the message was indeed sent by this party. In some embodiments, ifa secondary communication channel has been established between thefiltering system and the apparent sender, then this is used. Forexample, if the filtering system has access to a cell phone numberassociated with the sender, then an SMS or an automated phone call maybe generated, informing the sender that if he or she just sent an emailto the recipient, then he/she needs to confirm by responding to the SMSor phone call, or performing another confirming action, such as visitinga website with a URL included in the SMS. In some embodiments, thereceived email is identified to the recipient of the SMS/phone call,e.g., by inclusion of at least a portion of the subject line orgreeting. If no secondary communication channel has been established,then in some embodiments, the system sends a notification to the senderrequesting this to be set up, e.g., by registering a phone number atwhich SMSes can be received, and have this validated by receiving amessage with a confirmation code to be entered as part of the setup. Insome embodiments, to avoid spoofing of the system, the request is madein the context of an email recently sent by the party requested toregister. For example, the registration request may quote the recentlysent email, e.g., by referring to the subject line and the recipient,and then ask the sender to click on a link to register. Optionally, thissetup can be initiated not only for high-risk messages, but also as auser is qualified as trusted (e.g., having been detected to be afriend), which allows the system to have access to a secondarycommunication channel later on. Phone numbers can also be obtained bythe filtering system accessing address books of users who are protectedby the system, extracting phone numbers from emails that are beingprocessed, and associating these with senders, or other techniques.Other secondary channels are also possible to use, such as alternativeemail addresses, Skype messaging channels, Google Chat messages, etc. Inan alternative embodiment, it is possible to transmit an email messageto the sender of the high-risk message in response to the processing ofthe high-risk message, requiring the sender of the high-risk message toconfirm that this was sent by him or her by performing an action such asresponding to an identification challenge, whether interacting with anautomated system or an operator. This can be done on the same channel asused by the sender of the message, or to another email address, if knownby the system. Any identification challenge system can be used, asappropriate. This can be combined with the setup of a secondary channel,as the latter provides a more convenient method to confirm thetransmission of messages.

In some embodiments, the technique for quarantining high-risk messagessent by trusted parties until a secondary channel confirmation has beenreceived seamlessly integrates second factor authentication methods withdelivery of sensitive emails, such as emails containing invoices orfinancial transfer requests. This can be beneficial in systems that donot focus on blocking of high-risk messages as well as in systems suchas that described in the exemplary embodiment below.

In some embodiments, configured to protect consumers, content analysiswould not focus on mention of the word “invoice” and similar terms ofhigh risk to enterprises, but instead use terms of relevance to consumerfraud. For example, detection of likely matches to stranded travelerscams and similar can be done using a collection of terms or usingtraditional machine learning methods, such as Support Vector Networks(SVNs). In some embodiments, if a likely match is detected, this wouldinvoke a second-factor authentication of the message.

The use of second factor authentication (2FA) for confirmation isbeneficial to avoid risk. For example, if Alice sends an email to herbroker, Bob, asking Bob to sell some of her stock, then it is beneficialfor Bob to confirm with Alice before performing the sale. This avoidsperforming transactions as a result of attacks, such as a spoofingattack in which Eve is sending a spoofed message to Bob, appearing tocome from Alice. It also mitigates the threat associated with malwareand stolen computers. For example, consider a setting where Eve placesmalware on Alice's computer, causing an email to be sent from Alice toBob, in which Bob is asked to sell some of Alice's stock. In theseexamples, using a 2FA for confirmation reduces the threat, as if Evedoes not have the ability to receive the 2FA request and respond to iton Alice's behalf, then the email request will be ignored by Bob. The2FA confirmation requests can include SMS messages or (manually orautomatically placed) phone calls. Existing systems for sending 2FAconfirmation requests are not automated. Instead, for example, Bob readshis email from Alice, and determines in a case-by-case basis whether toinitiate a 2FA confirmation request. Sometimes, Bob may make a mistakeor be hurried by a high-priority request, thereby deciding to ignore the2FA confirmation. Scammers may trick Bob to omit the request. In someembodiments, the techniques described herein include automating thedetermination of when to send a 2FA confirmation request, and integratesthe confirmation with the delivery of the email. This way, Bob will notreceive the email from Alice until Alice has confirmed it, unless it isan email that does not require a confirmation, in which case it will bedelivered immediately.

In some embodiments, the techniques described herein are usable toautomate the use of 2FA for confirmation of emails associated withheightened risk. In some embodiments, this is a three-stage process, anexample of which is provided below.

In the first stage, channel information is obtained. In someembodiments, this channel information is a phone number of a party,where this phone number can be used for a 2FA confirmation. For example,if the phone number is associated with a cell phone, then an SMS canlater be sent for 2FA, as the need arises to verify that an email wassent by the user, as opposed to spoofed or sent by an attacker from theuser's account. Whether it is a cell phone number or landline number,the number can be used for placing of an automated phone call. Thechannel can also be associated with other messaging methods, such as IMor an alternative email address. In one embodiment, the first stage isperformed by access of records in a contact list, whether uploaded by auser of a protected system, by an admin associated with the protectedsystem, or automatically obtained by the security system by finding thecontact list on a computer storage associated with the protected system.Thus, in this embodiment, the setup associated with the first stage isperformed by what will later correspond to the recipient of an email,where the recipient is a user in the protected organization. In anotherembodiment, the first stage is performed by the sender of emails, i.e.,the party who will receive the 2FA confirmation request as a result ofsending a high-risk email to a user of the protected system. In oneembodiment, sender-central setup of the 2FA channel is performed afterthe sender has been identified as a trusted party relative to one ormore recipients associated with the protected system, and in someembodiments, is verified before being associated with the sender. Thisverification can be performed using standard methods, in which a code issent, for example, by SMS or using an automated phone call, to a phonenumber that has been added for a sender account, and after theassociated user has received the code and entered it correctly for thesystem to verify it, then the number is associated with the sender. If asender already has a channel associated with his or her email address,for example, by the first stage of the process having been performed inthe past, relative to another recipient, then in some embodiments, it isnot required to perform the setup again. If later on, a 2FA confirmationrequest fails to be delivered, then, in some embodiments, the channelinformation is removed and new channel information requested. Channelinformation can be validated by sending a link to an email accountassociated with a sender, containing a link, and sending a message witha code to the new channel, where the code needs to be entered in awebpage associated with the link in the email. In one embodiment, thisis performed at a time that there is no suspicion of the email accountbeing taken over. Alternatively, the validation can be performed by therecipient entering or uploading channel data associated with a sender.While the validation of the channel may not be completely full-proof,and there is a relatively small potential risk that an attacker wouldmanage to register and validate a channel used for 2FA, the typical casewould work simply by virtue of most people not suffering accounttake-overs most of the time, and therefore, this provides security forthe common case.

An alternative approach to register a channel is to notify the userneeding to register that he or she should call a number associated withthe registration, which, in some embodiments, includes a toll-freenumber, and then enter a code that is contained in the notification. Forexample, the message could be “Your email to Alice@company.com withsubject line ‘March invoice’ was quarantined. To release your email fromquarantine and have it delivered, please call <number here> and enterthe code 779823 when prompted.” In some embodiments, at any time, onecode is given out to one user. When a code is entered, the phone numberof the caller is obtained and stored. An alternative approach is torequest an SMS. For example, the message could be “Your email toAlice@company.com with subject line ‘March invoice’ was quarantined. Torelease your email from quarantine and have it delivered, please SMS thecode 779823 to short code <SMS number here>.”

In some embodiments, if the phone number has previously been used toregister more than a threshold number of channels, such as more than 10channels, then a first exception is raised. If the phone number isassociated with fraud, then a second exception is raised. If the phonenumber is associated with a VoIP service, then a third exception israised. If the phone number is associated with a geographic regioninconsistent with the likely area of the user, then a fourth exceptionis raised. Based on the exceptions raised, a first risk score iscomputed. In addition, in some embodiments, a second risk score iscomputed based on the service provider, the area code of the phonenumber, the time zone associated with the area code, the time of thecall, and additional aspects of the phone number and the call. In someembodiments, the first and the second risk scores are combined, and theresulting value compared to a threshold, such as 75. In someembodiments, if the resulting value exceeds the threshold, the risk isconsidered too high, otherwise it is considered acceptable. If the riskis determined to be acceptable, then in some embodiments, the phonenumber is recorded as a valid channel. If later it is determined that avalid channel resulted in the delivery of undesirable email messages,then in some embodiments, the associated channel data is removed orinvalidated, and is placed on a list of channel data that is associatedwith fraud.

In the second stage, a high-risk email is sent to a user of a protectedorganization, from a sender that the system determines is trusted to therecipient. In one embodiment, the email is placed in quarantine and a2FA confirmation request to the email sender is automatically initiatedby the security system, where the sender is the party indicated, forexample, in the ‘from’ field of the email. In some embodiments, this 2FAconfirmation is sent to the channel registered in the first stage. Inone embodiment, if this transmission fails, then a registration requestis sent to the email address of the sender of the email, requesting thatthe sender registers (as described in the first stage, above.

In a third stage, a valid confirmation to the 2FA confirmation requestis received by the system and the quarantined message is removed fromquarantine and delivered to the intended recipient(s). In the case wherea registration request was sent in the second stage, in someembodiments, a different action is taken, to take into account that thenew registration information may be entered by a criminal. An exampleaction is to remove the quarantined message from quarantine, mark it upwith a warning, the entered channel information, and a suggestion thatthe recipient manually verifies this channel information before actingon the email. The marked-up email can also contain a link for therecipient to confirm that the entered channel information is acceptable,or to indicate that it is not. If the system receives a confirmationfrom the recipient that the entered channel information is acceptablethen this information is added to a record associated with the sender.The email is then transmitted to the intended recipient(s).

An alternative authentication option is to request the senderauthenticate through a web page. A request with a URL link can be senton a variety of channels including the original sending email address,an alternate email address, or an SMS containing a URL. The appropriatechannel can be selected based on the likelihood of risk. A long randomcustom URL can be generated each time to minimize the likelihood ofguessing by an attacker. The user can click on the link and betransparently verified by the device information including browsercookies, flash cookies, browser version information or IP address. Thisinformation can be analyzed together to confirm that it is likely apreviously known device. For example, if there is no prior cookie andthe IP address is from another country, then this is unlikely to be thecorrect user. A second factor, in addition to device information, can bethe entry of a previously established passcode for the user. The secondfactor can be a stronger factor including a biometric, or token thatgenerates unique time based values. FIDO (Fast Identity Online)authentication tokens can be used to provide strong factor with a gooduser experience.

One authentication option is to reply with an email and ask the receiverto call a number to authenticate. This is an easy way to capture newphone numbers for accounts. Because the incoming phone number can beeasily spoofed, a follow up call or SMS back to the same number cancomplete the authentication. In one scenario, the user can be asked whatfollow up they would like. For example, “Press 1 to receive an SMS,Press 2 to receive a phone call.”

Authentication using a previously unknown phone number can also beperformed. For example, authentication can be strengthened by performingvarious phone number checks including a Name-Address-Phone (NAP) checkwith a vendor or a check against numbers previously used for scams or acheck against a list of free VOIP numbers.

Yet another example technique for 2FA involves hardware tokensdisplaying a temporary pass code. In some embodiments, the systemdetects a high-risk situation as described above and sends the apparentsender an email with a link, requesting that the apparent sender clickson the link to visit a webpage and enter the code from the 2FA tokenthere. After this code has been verified, in some embodiments, thehigh-risk email is removed from quarantine and delivered to therecipient. In this context, a second channel is not needed, as the useof the token makes abuse by a phisher or other scammer not possible.

Other conditional verification techniques can be conditionally used forhigh-risk situations involving emails coming from trusted accounts. Oneof the benefits of the techniques described herein is to selectivelyidentify such contexts and automatically initiate a verification, whileavoiding to initiate a verification for other contexts.

In one embodiment, the conditional verification is replaced by a manualreview by an expert trained in detecting scams. In some embodiments, theemail under consideration is processed to hide potential personallyidentifiable information (PII) before it is sent for the expert toreview. In some embodiments, at the same time, the email is placed inquarantine, from which it is removed after the expert review concludes.If the expert review indicates that the email is safe then, in someembodiments, it is delivered to its intended recipients, whereas if theexpert review indicates that it is not desirable, then it is discarded.

When the terms “blocked” and “discarded” are used herein, they areinterchangeably used to mean “not delivered”, and in some embodiments,not bounced to the sender. In some instances, a notification may be sentto the sender, explaining that the email was not delivered. The choiceof when to do this is, in some embodiments, guided by a policy operatingon the identified type of threat and the risk score of the email.

The benefits of the technology can be understood by looking at howdifferent attacks are addressed, and the extent to which they—shouldthey not be addressed—appear as desirable traffic. Examples of the maintypes of attack include: spoofed emails, account take-overs, deceptivedomains or email addresses, high-risk content, and other. Examples ofthe main types of desirable email include email from trusted parties(whether what we refer to as ‘friends’ or ‘internal’ traffic), and emailfrom parties that are not trusted. The associated relationships areconsidered in detail below:

-   -   Spoofed adversarial traffic is potentially likely, based, for        example, on observations of abuse attempts. In some embodiments,        the disclosed system detects (virtually) all spoofed adversarial        traffic, based, for example, on analysis of trust relationships        and inclusion of reply-to addresses. False positives based on        the analysis of trust relationships and reply-to addresses may        be very unlikely, but possible. To mitigate the risk of false        positives, in some embodiments, the system initiates a to-sender        verification request, as described below. From a practical        perspective, the disclosed approach can make the error rates        associated with spoofed emails negligible.    -   Adversarial traffic relying on account take-overs is also        potentially likely, based, for example, on observations of abuse        attempts. Such traffic may resemble benevolent traffic closely,        making it difficult to act on without causing errors. The        techniques described herein address this by, for example,        identifying high-risk messages based at least, among other        things, on content, quarantining such high-risk messages, and        conditionally releasing them from quarantine based either on a        valid response to a second factor authentication of the message        or on a request from the recipient to remove the message from        quarantine. In some embodiments, this is only done for high-risk        traffic, determined at least in part based on message content,        and therefore, avoids unnecessary actions for senders and        receivers of common low-risk traffic. Again, from a practical        perspective, the approach described herein can make the error        rates associated with emails arising from account take-overs        negligible.    -   The most common type of adversarial traffic, based on        observations of abuse attempts, corresponds to emails that are        sent from domains or accounts created by the attacker, and where        these use deceptive naming, whether of display names, user        names, domain names, or a combination of these. Using the        techniques described herein, such traffic can be made to stand        out from benevolent traffic. In some embodiments, this type of        traffic is detected by analyzing trust relationships and        determining whether the sender names are deceptive. In some        embodiments, this determination is based at least in part on        information relating to who is a trusted party to the recipient,        which in turn can be based on previously received and sent        emails, but also based on what is referred to herein as        “universal” trust relationships. An example of the latter is a        trust relationship a user may have with a famous brand, such as        a bank, based knowledge about the brand. For an email that is        identified as coming from a source with a very high deceptive        score, the email, in one embodiment, is blocked, whereas an        email coming from a source with a deceptive score that is not        very high but also not low can be marked up with a warning or        quarantined. This selection can be based on a configuration made        by the recipient or an admin associated with the recipient. The        approach described herein makes the error rates associated with        emails from deceptively named senders very low. Moreover, since        benevolent traffic from such sources is typically rare, the risk        of mis-classificaton is very low. In particular, false positives        are unlikely to be associated with trusted senders, since a        filtering action depends both on not coming from a trusted        source and being sent from a deceptively named account.    -   Another example type of adversarial traffic is different from        the three above-described types of adversarial traffic, and is        from an account controlled by the attacker, where this account        does not have a trusted status with respect to the recipient of        the email, and is not a whitelisted brand, such as a bank or a        company for which attackers cannot easily generate messages.        This excludes what are referred to as promiscuous domains, which        correspond to services where it is may be easy for an attacker        to register an account. The sender, furthermore, is typically        not deceptively named, but has high-risk content. Examples of        such content include keywords and phrases indicative of common        scams. The content portion can include text in the email, text        in attachments, and names of attachments. In some embodiments,        it also includes text associated with webpages hyperlinked from        the content portion, the associated URLs, and the functionality        of the webpages. One type of functionality is at least one text        entry field for which entered text is not represented as the        text itself, but as other characters, such as stars. This can be        common for password entry. In some embodiments, when the system        identifies an email that satisfies these criteria, it blocks the        email if the content risk is determined to be very high. In one        embodiment, it marks up emails that are determined not to be        very high risk, but also not low risk, or alternatively, places        such emails in quarantine. The decision of what action to take        on such messages can either be a user configuration selection,        an admin configuration selection, or a selection made as part of        the system design. The error rates associated with message        classifications of this type are typically low. This is because        messages from strangers, where these messages contain high-risk        content, are typically dangerous. In some embodiments, messages        from trusted parties are not considered in this category.    -   Yet another type of email is not from a trusted party, and does        not contain high-risk content. In some embodiments, such emails        are delivered in the recipient's inbox. In one embodiment, all        emails from parties who are not trusted are marked up with a        notification, such as “This email comes from a party with whom        you do not have a trust relationship.” In another embodiment,        such a warning is only added to messages whose risk exceeds a        minimum value (e.g., by coming from a newly registered domain or        having at least one word associated with risk in the content        portion). Messages of this type are typically of low actual        risk, and therefore, are safe to deliver to the recipient. There        is no risk associated with false positives, as no messages of        this type are blocked.

In some embodiments, if a message is determined to have a high risk ofbeing the result of a spoofing attack, a message of a first type ofmessage is transmitted to an address associated with the sender, whereasif a message is determined to have a high risk of being the result of anaccount take-over, then in some embodiments, a second type of message istransmitted to an address associated with the sender. In someembodiments, the classification of the problem is used in the selectionof the messaging method. In the first case, when there are indicationsthat the email is the result of a spoofing attack, then, in oneembodiment, a message is sent to the apparent sender of the email (butnot to the reply-to address, if such an address is present). The messagecan state, for example, “Your message with subject <subject line here>,which you sent to <recipient list here> has been quarantined. In orderto cause it to be delivered, please click here <hyperlink inserted here>or respond “ok” to this notification to confirm. By clicking orresponding, your email will be delivered. If you did not send the email,you do not have to do anything.” Note that if the message was spoofed,which means that it was sent by a party other than the claimed sender,then the apparent sender will not respond to the request, and therefore,the email associated with high risk would not be delivered.

In contrast, when an email is determined to have a high risk of beingassociated with an account take-over, then in some embodiments, a 2FAconfirmation request is initiated. This can include a message sent to anaddress other than the apparent sender, and may be a secondary emailaddress, a phone number or an instant messaging address. The content ofthe notification message may be similar to what was described in thecontext of suspected spoof messages. If no valid channel address hasbeen registered, in some embodiments, the recipient receives a messagedescribing that the email has been placed in quarantine, but no messagewould be sent to an account associated with the apparent sender.

In some embodiments, if an email is placed in quarantine and not movedfrom there by an action of a sender or the recipient, then after athreshold duration has passed, it is be erased. This threshold can forexample be one week, one month, forever, or any other appropriatethreshold time period.

In cases where it is determined that an email is either at high risk forbeing associated with spoofing or with an account take-over, but itcannot be determined whether it is one or the other, then one exampleresponse is to verify whether the apparent sender is associated with avalid channel, and if so, send a message over that channel; andotherwise, to send a message to the apparent sender. In someembodiments, in the second case, this request also involves theregistration and validation of a channel. If a message can be determinedto almost certainly be the result of spoofing, for example, by analyzingthe route and finding anomalies indicative of spoofing, then no requestis sent, but the message is simply blocked. Similarly, if a message canbe determined to almost certainly be the result of account take-over,such as exhibiting an anomalous volume of high-risk messages being sentfrom it, then no request is sent, but the message is simply blocked.

FIG. 1 is a block diagram illustrating an embodiment of a system fordetecting scam. In the example shown, system 100 may be used to detectscam such as business email compromise. As shown in this example, system100 includes interface 102, risk classification engine 104, quarantineengine 106, confirmation engine 108, risk data collection engine 110,risk data assessment engine 112, and database 114.

In this example, a message such as an email is received over a network(such as the Internet) via interface 102. The email message is passed torisk classification engine 104, which is configured to determine a riskassociated with the email message. In some embodiments, the risk isdetermined using the detectors and components described above. In someembodiments, classifying/assessing the risk associated with the emailmessage includes evaluating header and/or content portions of the emailmessage to determine whether the email message is indicative ofmalicious intent, such as spoofing, account takeover, or some other typeof scam. In some embodiments, as described above, the riskassessment/classification is based on determining whether the emailmessage is associated with a deceptive sender. Theclassification/assessment may also be performed based on trust-basedfiltering, as described above.

Based on the risk assessment, the message may be passed to quarantineengine 106. For example, if the risk determined for the message exceedsa threshold, then the message is placed in quarantine, and is prevented(e.g., at least temporarily) from being delivered.

Confirmation engine 108 is configured to request confirmation that thesender of the message did indeed originate the email message. In someembodiments, confirmation is obtained using second factor authentication(2FA). The manner in which the confirmation is sent may be determinedbased on contact information associated with the email address of thesender. For example, as described above, if a cellular phone number waspreviously associated with the email address, in some embodiments, 2FAbased on a text message (e.g., short message service (SMS) message) isperformed. In other embodiments, as described above, email based 2FA maybe performed (e.g., because SMS is not possible due to there not beingan associated phone number). In some embodiments,enrollment/registration may be performed as well, as described above.

Risk data associated with the performing of the 2FA is collected by riskdata collection engine 110. The collected data is then assessed usingrisk data assessment engine 112 and in some embodiments, stored todatabase 114. Based on the risk assessment using the collected dataassociated with the confirmation, a determination is made whether or notto deliver the email message to the recipient.

In some embodiments, the scam detection system described hereincomprises standard commercially available server hardware (e.g., amulti-core processor, 4+ Gigabytes of RAM, and one or more Gigabitnetwork interface adapters) and runs typical server-class operatingsystems (e.g., Linux), as well as Java HTTP server software stack. Thescam detection system can be implemented using a scalable, elasticarchitecture and may comprise several distributed components, includingcomponents provided by one or more third parties. Further, when the scamdetection system is referred to herein as performing a task, such asstoring data or processing data, it is to be understood that asub-component or multiple sub-components of the scam detection system(whether individually or in cooperation with third party components) maycooperate to perform that task.

FIG. 2A is a flow diagram illustrating an embodiment of a process fordetecting scam. In some embodiments, process 200 is executed by system100 of FIG. 1 . The process begins at 202 when a sender, having a firstemail address, is associated with a set of secondary contact data items.Examples of secondary contact data items include a (cellular) phonenumber, a second email address, an instant messaging identifier, or anyother appropriate contact data item.

At 204, it is determined that an email message purporting to originatefrom the sender's first email address has been sent to a recipient. Insome embodiments, a risk is determined to be associated with the emailmessage, for example, using the message risk evaluation andclassification described above. At 206, prior to allowing access by therecipient to the email message, it is requested, using at least onesecondary contact item in the set of secondary contact data items, thatthe sender confirm that the email message was indeed originated by thesender. For example second factor authentication is performed to verifyor confirm that the sender did originate the email message. In someembodiments, the at least one secondary contact item is associated witha secondary communication channel. For example, the request may be madeusing SMS and/or email. At 208, in response to receiving a confirmationfrom the sender that the sender did originate the email message, theemail message is delivered to the recipient.

FIG. 2B is a flow diagram illustrating an embodiment of a process fordetecting scam. In some embodiments, process 230 is executed by system100 of FIG. 1 . The process begins at 232 when a first display name isassociated with a first email address. At 234, it is determined that anemail message purporting to originate from a second email addressincludes, in a header, the first display name or a second display namedetermined to be similar to the first display name. At 236, it isdetermined that a risk associated with delivery of the email message toa recipient exceeds a threshold. At 238, prior to allowing access by therecipient to the email message, a confirmation is requested via arequest email transmitted to the second email address. At 240, inresponse to receiving the confirmation, the email message is deliveredto the recipient.

FIG. 2C is a flow diagram illustrating an embodiment of a process fordetecting scam. In some embodiments, process 260 is executed by system100 of FIG. 1 . At 262, a first display name is associated with a firstemail address. At 264 it is determined that an email message,originating from a second email address that is different from the firstemail address, includes, in a header, the first display name or a seconddisplay name determined to be similar to the first display name. At 266,prior to allowing access by the recipient to the email message, aconfirmation is requested via a request email transmitted to the firstemail address. At 268, in response to receiving the confirmation, theemail message is delivered to the recipient.

FIG. 2D is a flow diagram illustrating an embodiment of a process fordetecting scam. In some embodiments, process 290 is executed by system100 of FIG. 1 . At 292, it is determined that a sender of an emailmessage that has been sent to a recipient is not trusted with respect tothe recipient.

At 294, a first set of data including at least one of an email addressand a display name associated with the not-trusted sender of the emailmessage is compared with a second set of data including at least one ofan email address and a display name associated with a trusted senderthat is trusted with respect to the recipient. In various embodiments,the trusted sender includes at least one of a friend, an internal party,a party included in an entry in an address book associated with therecipient, a party connected to the recipient on a network, and a partythat has previously communicated with the recipient via a messagingapplication. In some embodiments, comparing the first and second sets ofdata is performed with respect to at least one of display name, username, domain name, and top level domain (TLD).

At 296, based at least in part on the comparison, it is determined thata risk associated with delivery of the email message to the recipientexceeds a threshold. At 298, an action is performed in response todetermining that the risk associated with delivery of the email messageto the recipient exceeds the threshold. Examples of such actions includequarantining the email message, including a portion of the email messagein a request, modifying the email message, and marking the email messagewith a warning.

The following are additional example embodiments of the scam detectiontechniques described herein:

In some embodiments, detecting scam email includes the use of at leasttwo deception detection components and a combining logic componentconfigured to match outputs of the at least two deception detectionmechanisms with at least one scammer strategy. In some embodiments, whenan input email is evaluated, a filtering decision is generated based onthe output of the combining logic component. In some embodiments, atleast one deception detection component uses data relating to emailaddresses in the headers of the input email, and at least one deceptiondetection component may use data associated with the recipient of theinput email.

In another example embodiment, detecting scam email includes the use ofat least two deception detection components and a combining logiccomponent that is non-monotonically increasing. In some embodiments, anemail classification decision is generated by evaluating the at leasttwo deception detection components on the email, and combining theoutputs of the at least two deception detection components using thecombining logic component.

In some embodiments, determining trust includes the use of a trafficscan unit and a classification unit. In some embodiments, the trafficscan unit is configured to scan email traffic and determine, based onstored criteria and the scanned traffic, that a first sender qualifiesas trusted to a first receiver. In some embodiments, after this has beendetermined, the traffic scan unit is further configured to generate andstore an approval, where the approval includes information about thefirst sender and a time stamp. In some embodiments, the classificationunit is configured to read the approval and determine whether apre-configured amount of time has elapsed since the approval wasgenerated. In some embodiments, a classification is conditionallygenerated when this is determined to have taken place, where theclassification indicates that the first sender is trusted (e.g., to thefirst receiver or users associated with the first receiver).

In some embodiments, a trusted sender is enrolled in a secondarycommunication channel. Enrolling the trusted sender in a secondarycommunication channel may include identifying a high-risk message fromthe trusted sender, placing the high-risk message in quarantine andgenerating a request on the secondary communication channel, followed bydelivering the high-risk message to its recipients conditional on theresponse to the request.

In some embodiments, high-risk messages sent from trusted senders arequarantined. Quarantining high-risk messages sent from trusted sendersmay include sending a 2FA confirmation request to a validated channelassociated with the sender, where the email is moved from the quarantineto the inbox of the recipient conditional on a valid response to therequest.

In some embodiments, a message is classified as being associated with atleast one of a high risk of spoofing, a high risk of account take-over,a high risk of deceptive name usage, and a high risk based on content.An action may be performed, where the action associated with the messageclassified as being associated with a high risk of spoofing may be afirst type of request automatically sent to the address of the sender ofthe message, and where the action associated with the message classifiedas being associated with a high risk of account take-over is a secondtype of request automatically sent to an address associated with thesender of the message, but distinct from the address of the sender ofthe message. In some embodiments, the message is delivered to therecipient conditional on receiving a valid response to the request.

Exemplary Embodiment

In the following, the techniques described herein are described usingexample pseudocode associated with an example implementation. Theexample embodiment is provided for illustrative purposes, andalternative embodiments are possible.

The following embodiment uses a data structure such as the following:

Example Data Structure:

In this example, each email E is represented by the following components

-   -   E.from: account (see below for a description)    -   E.sender: account    -   E.replyto: account    -   E.to: account    -   E.content: a pointer to a string storage area    -   E.attachments: a pointer to a linked list of pointers to        attachments

In turn, an account A is represented in the following way:

A.displayname % This corresponds to the underlined part in an emailaccount % “Joe Schmoe” <JoeS@hiscompany.com> A.username % “Joe Schmoe”<JoeS@hiscompany.com> A.domainhead % “Joe Schmoe” <JoeS@hiscompany.com>A.TLD % “Joe Schmoe” <JoeS@hiscompany.com>

-   -   % From those, one can construct the following useful        combinations:    -   % address:=A.username+“@”+A.domainhead+“.”+A.TLD    -   % domain:=A.domainhead+“.”+A.TLD    -   % addresshead:=A.username+“@”+A.domainhead    -   % account:=A.displayname+‘        ’+A.username+“@”+A.domainhead+“.”+A.TLD

Furthermore, in this example, a user or a set of users is associatedwith a contact list C, comprising entries Ci. The entries Ci can berepresented in the following way:

-   -   Ci.A: account    -   Ci.NumberReceiptsFrom: a counter    -   Ci.NumberEmailsTo: a counter    -   Ci.DateQualified: a time stamp    -   Ci.friend: a boolean    -   Ci.RecordedReplyto: a list of reply-to addresses that have been        used by Ci.A

The above are example data structure components, provided forillustrative purposes.

Example Deception Detectors:

The following description details an example set of deception detectors,each one of which is associated with the detection of scams, and BECscams in particular:

HasReplyTo

input: an email E

output: a boolean

process:

return (E.ReplyTo!=empty field) and((E.ReplyTo).address!=(E.from).address)

why: for many BEC scams, the use of reply-to is central

HowManyRecipients

-   -   input: an email E, protected recipient email account A    -   output: an integer value    -   process: returns how many people in to/cc fields are protected        accounts    -   a global variable: the protected domain D    -   detailed process:    -   Create an empty set S.    -   For all the recipients R in the to-field and all the recipients        in the cc-field:        -   If (R.domain!=D) and (R!=A) then            -   S:=S+R    -   Return(length(S)) % that is, how many elements are in S

why: for most BEC scams, there is exactly one email recipient in theenterprise—the scammer does not want to encourage discussion!

In one embodiment, accounts with the vacation auto-reply set are notcounted, but otherwise, the same process as described above isperformed. Similarly, in some embodiments, unattended email address arenot counted; these are email addresses that cause automated responses,or where a human user is rarely reviewing the incoming traffic, or onlywith a substantial delay, such as several weeks. In some embodiments,facts like these are automatically inferred by the system by observingincoming and outgoing email traffic.

DeceptiveCompare

-   -   input: an account A1, an account A2    -   output: an integer value (0-100)

process: returns how deceptive an account is relative to another. Notethat if the corresponding addresses are identical, that is not deceptiveat all.

-   -   detailed process:

  % The algorithm compares: %  two input accounts %  two inputaddressheads %  two input addresses %

-   -   % In an alternative embodiment, the following are also compared:

%  two input domains %  two input domainheads %  two input persons IfA1.address=A2.address then    Return(0) % They are not deceptive if theemail addresses are identical    else   Return(trunc(100*max(  JW(A1.account, A2.account),          JW(A1.addresshead, A2. addres shead),          JW(A1.address, A2.address))))

HowDeceptiveIsSender

-   -   input: an email E, contact list C    -   output: an integer value (0-100)    -   process: returns how deceptive a sender is, relative to        recipients contacts

In some embodiments, senders are deceptive if they are similar tocontacts. (In contrast to reply-to addresses, which are deceptive ifthey are similar to the from field.)

-   -   process detail:    -   MaxDeceptive:=0    -   For all entries Ci of C:        -   MaxDeceptive:=Max(MaxDeceptive, DeceptiveCompare(E.from,            Ci.A))    -   Return(MaxDeceptive)

why: Many BEC attacks involve the use of sending accounts that make therecipient believe that they know the sender.

HowDeceptiveIsReplyTo

-   -   input: an email E    -   output: an integer value (0-100)    -   process: returns how deceptive a reply-to address is, relative        to from field & sender field    -   Reply-to addresses are deceptive if they are similar to (but not        the same as) the from field. (In contrast to senders, which are        deceptive if they are similar to a contact.)    -   process detail:    -   Return(DeceptiveCompare(E.replyto, E.from))

why: Some BEC scams (e.g., those involving spoofing or account-takeover(ATO)) come from “trusted” accounts; most other scams typically comefrom people with limited interaction history. Taking this structuralapproach into consideration—along with other features that characterizethe cases—allows for the identification of common cases without highrisks for misclassification.

IsFriend

input: an email E, contact list C output: a boolean process: return trueif E is a friend of the party with contact list C process detail:    Ifthere is a record Ci of C such that Ci.A=E.account then      return(Ci_friend)    else       return(false)

why: Many BEC scams (such as those based of spoofing) need a response toan address other than the apparent sending address—but want these tolook similar.

UnFriend

      input: an email address A, contact list C       output: N/A      process: remove an entry from the friend list and thesoon-to-be-friend list       process detail:          If there is arecord Ci such that Ci.A=A then             Ci_friend:=false         %not a friend (if he were)             Ci.NumberReceiptsFrom:=0  %restart counters             Ci.NumberEmailsTo:=0    % restart counters            Ci.DateQualified:= nil       % set to “not qualified”         % Note that the record stays, but the “friend” designation isset to false

why: When an obvious scammer is identified, this party should beunfriended. In such a scenario it is not necessarily the case that emailwill not be delivered—in some embodiments, that “nasty” email will moreeasily get trapped. If somebody who was ATOed were to be unfriended, andthen had bad email sent from their account, there is potentially verylimited damage: as soon as they recover their account, they will startcommunicating as usual, and soon enough, they will be back on the friendlist.

IsInternal

-   -   input: an email address A, recipient domain D    -   output: a boolean    -   process: returns whether A is internal to the recipient    -   process detail:    -   return(A.domain=D)

why: Some BEC scams (e.g., those involving spoofing or ATO) come from“trusted” accounts; typically, most other scams come from people withlimited interaction history. Taking this structural approach intoconsideration—along with other features that characterize thecases—allows for the identification of common cases without high risksfor misclassification.

IsChameleon

input: an email E output: a boolean process: returns whether A ischameleon process detail: If length(E.username)>ChameleonLengthThresholdthen else if (E.from in ChameleonList) then    return(true) else   return(false)

Here, ChameleonLengthThreshold=30 is an example of a parameter choice.

In some embodiments, ChameleonList is a relatively short list of themost common senders of chameleon email, such as member@linkedin.com,*@yahoogroups.com, *@googlegroups.com, where * denotes a wildcard.

In one embodiment, the list ChameleonList is generated as follows:

1. A screening component observes reply-to addresses for all analyzedemail. For each protected account, it records reply-to addresses usedfor all friends of the protected account. (where friends can include atrusted sender, as described above). In some embodiments, this is onlydone for emails that were considered safe.

2. If the number of observed reply-to addresses for one sender and oneprotected account exceeds a threshold (such as 10, which may be the sizeof the vector we use to store reply-to addresses for each senderaccount) then this sender is considered a chameleon reply-to sender. Forexample, a chameleon sender such as jobs@newopenings.com might havemultiple reply addresses like reply1492A@newopenings.com . . .reply2201z.com to track their email responses. In some embodiments, aflag is set to identify this.

3. Periodically, and in some embodiments, in batch mode, a componentscans the observed reply-to addresses for all protected accounts, anddetermines how many unique reply-to addresses there are for each uniquesender. In some embodiments, if this exceeds a tunable threshold (say100), then this sender is considered a chameleon reply-to sender. Insome embodiments, a second flag is set to identify this. It can be adesign option whether to have one flag per protected account (which maycreate challenges in updating) or one global record with a flag. Thisprocess can also be performed continuously, as a new incoming oroutgoing email is processed.

why: Many legitimate merchants and newsletters use reply-to to track theefficacy of their emails. Many benevolent email senders use reply-toheavily. To save effort, storage, and to reduce error rates, it would bebeneficial to avoid paying attention to these.

IsAssociatedReplyTo

-   -   input: a contact list C, an email E    -   output: a boolean    -   process: returns whether the reply-to of E has been used before        by the same sender        -   also sets a global boolean variable to prompt conditional            addition to the RecordedReplyTo

process detail: response = false For all Ci in C  if (E.from = Ci.from)then   if (Ci.friend) then    if (E.replyto in Ci.RecordedReplyto) then    response=true    else     AddToRecordedReplyTo := true   % and wecan quit the loop “For all Ci in C” then return(response)

why: Some benevolent email senders may use reply-to, but most (exceptchameleon senders) typically use the same reply-to (or a small number ofthese) all the time. It would be beneficial to know if a reply-toaddress that is seen is “new”—e.g., that signals risk.

NowRecordReplyTo

-   -   input: a contact list C, an email E    -   output: none; modifies contact list being input    -   process: the email is safe, the sender has a new reply-to—record        it!    -   process detail:    -   Create a new Ci entry and add to C    -   % {circumflex over ( )} conditional on there being space, based        on a limited number of entries per record Ci    -   % For example, this limited number may be 10.    -   Ci.RecordedReplyto:=(E.replyto).address

why: In some embodiments, this provides a maintenance routine for“IsAssociatedReplyTo”.

Promiscuous

-   -   input: an email account A    -   output: a boolean    -   process: returns whether the address corresponds to a domain        where membership is not    -   detailed process:

In some embodiments, a list of known promiscuous organizations iskept—Gmail, Yahoo, etc. This may comprise the 100 most commonly seenpromiscuous organizations. In an alternative embodiment, a list of knownnon-promiscuous organizations that are found to otherwise causemisclassifications is also kept.

If A in KnownPromiscuous then    Promiscuous:= true else if A inKnownNonPromiscuous then    Promiscuous := false else    Promiscuous :=Age(Domain(A)) < DomainAgeThreshold    % Heuristic to mistrust newdomains    % Here DomainAgeThreshold may be 1 month    % This is just anexample heuristics.

why: Some email accounts may be easy for criminals to create, others maynot. Being able to determine what type of account is associated with anemail facilitates the determination of whether the email is high risk ornot.

ReplyToDifferentDomain

-   -   input: an email E    -   output: a boolean    -   process: returns whether the reply-to is from a different domain        than from/sender    -   process detail:    -   Return((E.replyto).domain!=(E.from).domain)

why: If the reply-to from an email sent by a user of a protectedenterprise goes to the same enterprise, that is lower risk than if thereply-to goes to another domain.

PotentialPhishingURLs

        input: en email E         output: a boolean         process:returns whether the content portion contains a likely password entryform         process detail:         response := false         ScanE.content.         For each hyperlink H of E.content:            Visitthe page H.            If the visited site               a. contains atleast two input fields               b. where one of them results instarred-out text upon entry            then            response:=true        Return(response)      Note: In some embodiments, this is analternative to ProtectPhishingURLs. In some embodiments, not both areneeded.

why: Detecting attempts to phish users of protected enterprises can bebeneficial.

ProtectPhishingURLs

-   -   input: an email E    -   output: none, but the function rewrites E    -   process: replaces all hyperlinks with “safe” alternatives    -   Note: In some embodiments, this is an alternative to        PotentialPhishingURLs. In some embodiments, not both are needed.    -   process detail:        -   Scan E.content.        -   For each hyperlink H of E.content:            -   Replace H with a proxy hyperlink PH (described below),In                some embodiments, the proxy hyperlink is used to:    -   1. The proxy hyperlink is hosted by a security organization or        the protected enterprise and encodes the “original” hyperlink.    -   2. When the proxy hyperlink is visited, it causes a wget, java        httpget or a spider (that is dressed up as a browser, and which        does not comply with robots.txt) to visit the original hyperlink        site.    -   3. If the visited site:        -   a. contains at least one input field        -   b. where one input field results in starred-out text upon            entry (in HTML this would be an form input field where the            type attribute would be ‘password’)        -   then            -   display a warning message—unsafe site, potential                phishing—with a link to proceed anyway—In some                embodiments, this link leads to the original site        -   else if the webpage contains content and/or logos for a            known brand but the URL does not correlate with the brand            -   then                -   display a warning message—unsafe site, potential                    phishing—with a link to proceed anyway—this link                    leads to the original site            -   else automatically redirect to the original site    -   Alternate embodiment:        -   1. The proxy hyperlink is hosted by the scam detection            system and encodes the “original” hyperlink.        -   2. Before a proxy hyperlink is visited, the suspect link can            be analyzed before the click or at the click. This:            -   a. allows the site to be checked in user time instead of                in real-time in the emails stream            -   b. performs the check when the user is about to access                the site. Scammers can vary the content and the click                time check is a better more timely content verification.                If there is problem a warning message is displayed “This                site may not be trusted. If you are asked to enter a                password, be very careful. Click to proceed.”        -   3. If there is no problem with the destination site, then in            some embodiments, the system provides a silent redirect to            the intended site when the user clicks.

Note: In some embodiments, this is an alternative toPotentialPhishingURLs. In some embodiments, not both are needed.

why: Detecting attempts to phish users of protected enterprises can bebeneficial.

ResetVariables

-   -   process:        -   HasReplyTo:=false        -   IsChameleon:=false        -   HowDeceptiveIsReplyTo:=0        -   IsAssociatedReplyTo:=false        -   Classification:=safe        -   AddToRecordedReplyTo:=true

why: In some embodiments, this is a maintenance routine for the combinglogic.

JW % this is an Example of an Improved Version of the Jaro-WinklerAlgorithm

-   -   inputs: two accounts    -   process:

Step 1: Normalization.

-   -   In one embodiment, the following normalization methods are        applied:        -   1. Identify homograph attacks.        -   If any sender has a display name, user name or domain name            that includes unicode characters matching a list of known            suspect characters intermixed with non-unicode characters,            then an action is taken, where this action is at least one            of flagging the email as high-risk, mapping the suspect            characters to corresponding characters that look similar;            and causing a risk score to be increased. For example,            PayPal can be spelled using Cyrillic ‘a’ characters while            the others could be Latin-1 characters.        -   2. Identify different components and normalize.        -   Typical display names consist of multiple “words” (i.e.,            names). These are separated by non-letters, such as commas,            spaces, or other characters. These are normalized, e.g., by            being sorted alphabetically.        -   3. Identify non-letters and normalize.        -   Anything that is not a letter is removed (while keeping the            “sorted words” separated as different components)    -   Then, in some embodiments, there is a comparison of the sorted        list of components to all similarly sorted lists associated        with (a) friends, (b) common brands, and (c) special words, such        as “IT support”. In some embodiments, this comparison is        approximate, and is detailed below.

Step 2: Comparison.

-   -   In some embodiments, a module compares two lists of components,        say (a1, a2) with (b1, b2, b3), and outputs a score.    -   Here, (a1, a2) may represent the display name of a friend e.g.,        (a1,a2)=(“Doe”,“John”), and (b1, b2, b3) the display name of an        incoming non-friend email, e.g., (b1,b2,b3)=(“Doe”, “Jonh”,        “K”).    -   Next, the module compares all friend-names to the name of the        incoming non-friend email. For each one, the following is done:        -   1. Compare one component from each list, e.g., compare a1            and b1, or a1 and b2.        -   2. Are two components the same? Add to the score with the            value MATCH, and do not consider this component for this            list comparison anymore.        -   3. Is the “incoming” component the same as the first letter            of the friend component? Add to the score with the value            INITIAL, but only if at least one “MATCH” has been found,            and do not consider this component for this list comparison            any more.        -   4. Is the similarity between two components greater than a            threshold (such as 0.8)? Then add to the score. potentially            weighted by the length of the string to penalize long            matching strings more than short matching strings) with the            value SIMILAR and do not consider this component for this            list comparison any more.        -   5. If there is any remaining components of the incoming            message, add to the score by the value MISMATCH, but only            once (i.e., not once for each such component)    -   If the resulting score is greater than a threshold MATCH, then        it is determined that there is a match.    -   Here are some example value selections:        -   MATCH=50        -   INITIAL=10        -   SIMILAR=30        -   MISMATCH=−20    -   In one alternative embodiment, the module sorts the components        within each list alphabetically, if not already done. It then        combines the components within a list by concatenating them.        After this is done, it uses a string comparison algorithm on the        resulted two concatenated results.

Comparing Strings

One example approach to compare strings is to use the Jaro-Winkleralgorithm, or a version thereof.

% In an alternative embodiment,

% * If two long strings are very similar, that is more deceptive

% than if two short strings are similar, and is given a higher score

% * If one of the addresses is a “famous” address (name of CEO or

%“Bank of America”) then that is more deceptive than otherwise,

% and is given a higher score

-   -   One possible string comparison algorithm is the following:

 package zapfraud;  public class DiffScore  {   // fromhttp://web.archive.org/web/20100227020019/http://www.census.gov/geo/msb/stand/strcmp.c  /* strcmp95.c Version 2                   */   /* The strcmp95function returns a double precision value from 0.0 (total   disagreement) to 1.0 (character-by-character agreement). The returned   value is a measure of the similarity of the two strings.     */   //#include <ctype.h>   // #include <string.h>   // #defineNOTNUM(c)  ((c>57) || (c<48))   static Boolean NOTNUM(char c) { return((c>57) || (c<48)); }   // #define INRANGE(c)  ((c>0) && (c<91))   static Boolean INRANGE(char c) { return ((c>0) && (c<91)); }   //#define MAX_VAR_SIZE 61   static final int MAX_VAR_SIZE=61;   // #defineNULL60 ″   static final char NULL60 = ‘\0’;   //char[ ][ ] sp = newchar[39][2];    /*    {‘A’,‘E’, ‘A’,‘I’, ‘A’,‘O’, ‘A’,‘U’, ‘B’,‘V’,‘E’,‘I’, ‘E’,‘O’, ‘E’,‘U’,    ‘I’,‘O’, ‘I’,‘U’, ‘O’,‘U’, ‘I’,‘Y’,‘E’,‘Y’, ‘C’,‘G’, ‘E’,‘F’,    ‘W’,‘U’, ‘W’,‘V’, ‘X’,‘K’, ‘S’,‘Z’,‘X’,‘S’, ‘Q’,‘C’, ‘U’,‘V’,    ‘M’,‘N’, ‘L’,‘I’, ‘Q’,‘O’, ‘P’,‘R’,‘I’,‘J’, ‘2’,‘Z’, ‘5’,‘S’,    ‘8’,‘B’, ‘1’,‘I’, ‘1’,‘L’, ‘0’,‘O’,‘0’,‘Q’, ‘C’,‘K’, ‘G’,‘J’,    ‘E’,‘’, ‘Y’,‘’, ‘S’,‘’};    */   Stringbase = “AAAABEEEIIOIECEWWXSXQUMLQPI2581100CGEYS”   String alt =“EIOUVIOUOUUYYGFUVKZSCVNIORJZSBILOQKJ ”;   int[ ][ ] adjwt;   publicDiffScore( )   {    int[ ][ ] adjwt = new int[91][91];    /* Initializethe adjwt array on the first call to the function only.     The adjwtarray is used to give partial credit for characters that     may beerrors due to known phonetic or character recognition errors.     Atypical example is to match the letter “O” with the number “0”  */   for (int i=0; i<91; i++) for (int j=0; j<91; j++) adjwt[i][j] = 0;   for (int i=0; i<36; i++)    {    adjwt[base.charAt(i)][alt.charAt(i)] = 3;    adjwt[alt.charAt(i)][base.charAt(i)] = 3;    }   }   // doublestrcmp95(char *ying, char *yang, long y_length, int *ind_c[ ])   doublescore(String ying, String yang, String option)   {   /* Arguments:   ying and yang are pointers to the 2 strings to be compared. Thestrings    need not be NUL-terminated strings because the length ispassed.    y_length is the length of the strings.    ind_c is an arraythat is used to define whether certain options should be    activated. Anonzero value indicates the option is deactivated.

-   -   The options are:

  ind_c[0] Increase the probability of a match when the number ofmatched     characters is large. This option allows for a little more    tolerance when the strings are large. It is not an appropriate    test when comparing fixed length fields such as phone and     socialsecurity numbers.   ind_c[1] All lower case characters are converted toupper case prior     to the comparison. Disabling this feature meansthat the lower     case string ″code″ will not be recognized as the sameas the     upper case string ″CODE″. Also, the adjustment for similar    characters section only applies to uppercase characters.   Thesuggested values are all zeros for character strings such as names. */  int pass = 0;   // int[ ][ ] adjwt = new int[91][91];   Stringying_hold = ″″;   String yang_hold = ″″;   char[ ] ying_flag = newchar[MAX_VAR_SIZE];   char[ ] yang_flag = new char[MAX_VAR_SIZE];  double weight, Num_sim;   int minv, search_range, lowlim,     hilim,N_trans, Num_com;   int  yl1, yi_st, N_simi;   int i, j, k;   /* Ifeither string is blank - return - added in Version 2     */   if(ying.isEmpty( )) return(0.0);   if (yang.isEmpty( )) return(0.0);   /*Identify the strings to be compared by stripping off all leading and   trailing spaces.*/   ying = ying.trim( );   yang = yang.trim( );   //strncat(ying_hold,&ying[yi_st],ying_length);   //strncat(yang_hold,&yang[j],yang_length);   ying_hold = ying;   yang_hold= yang;   if (ying.length( ) > yang.length( ))   {    search_range =ying.length( );    minv = yang.length( );   }   else   {    search_range= yang.length( );    minv = ying.length( );   }   /* If either string isblank - return         */   /* if (!minv) return(0.0);    removed inversion 2  */   /* Blank out the flags                  */   //ying_flag[0] = yang_flag[0] = 0;   //strncat(ying_flag,NULL60,search_range);   //strncat(yang_flag,NULL60,search_range);   search_range =(search_range/2) - 1;   if (search_range < 0) search_range = 0; /* addedin version 2   */   /* Convert all lower case characters to uppercase.      */   ying = ying.toUpperCase( );   yang = yang.toUpperCase();   /* Looking only within the search range, count and flag the matchedpairs. */   */Num_com = 0;   yl1 = yang.length( ) - 1;   for (i = 0;i <ying.length( );i++)   {    lowlim = (i >= search_range) ? i -search_range : 0;    hilim = ((i + search_range) <= yl1) ? (i +search_range) : yl1;    for (j = lowlim;j <= hilim;j++)    {     if((yang_flag[j] != ′1′) && (yang_hold.charAt(j) == ying_hold. charAt(i)))    {      yang_flag[j] = ′1′;      ying_flag[i] = ′1′;      Num_com++;     break;     }    }   }   /* If no characters in common -return         */   if (0 == Num_com) return(0.0);   /* Count the numberof transpositions         */   k = N_trans = 0;   for (i = 0;i <ying.length( );i++)   {    if (ying_flag[i] == ′1′)    {     for (j=kj <yang.length( );j++)     {      if (yang_flag[j] == 1)      {       k =j + 1;       break;      }     }     if (ying_hold.charAt(i) !=yang_hold.charAt(j)) N_trans++;    }   }   N_trans = N_trans / 2;   /*adjust for similarities in nonmatched characters*/   N_simi = 0;   if(minv > Num_com)   {    for (i = 0;i < ying.length( );i++)    {     if(ying_flag[i] == ′ ′ && INRANGE(ying_hold.charAt(i)))     {      for(j=0;j< yang.length( );j++)      {       int x = ying_hold.charAt(i);      int y = yang_hold.charAt(j);       if (yang_flag[j] == ′ ′ &&INRANGE(yang_hold.charAt(j)))       {        if (adjwt[x][y] > 0)       {         N_simi += adjwt[x][y];         yang_flag[j] = ′2′;        break;        }       }      }     }    }   }  Num_sim =((double) N_simi)/10.0 + Num_com;  /* Main weight computation.*/ weight= Num_sim / ((double) ying.length()) + Num_sim / ((double)yang.length( ))    + ((double) (Num_com - N_trans)) / ((double)Num_com);   weight = weight / 3.0;   /* Continue to boost the weight ifthe strings are similar    */   if (weight > 0.7)   {    /* Adjust forhaving up to the first 4 characters in common    */    j = (minv >= 4) ?4 : minv;    for (i=0;((i<j)&&(ying_hold.charAt(i)==yang_hold.charAt(i))&&(NOTNUM(ying_hold.charAt(i))));i++);    if (i >0) weight += i * 0.1 * (1.0 - weight);    /* Optionally adjust for longstrings.          */    /* After agreeing beginning chars, at least twomore must agree and     the agreeing characters must be > .5 ofremaining characters.   */    if ((option.contains(″ADJUST_LONG″)) &&(minv>4) && (Num_com>i+1) && (2*Num_com>=minv+i))    if(NOTNUM(ying_hold.charAt(0)))    {     weight += (double) (1.0-weight) *      ((double) (Num_com-i-1) / ((double) (ying.length( )+yang.length()-i*2+2)));    }   }   return(weight);  } } /* DiffScore */

Example Combining Logic:

The following is an example combining logic. ‘%’ is the start of acomment and ‘:=’ is an assignment statement in the logic below. Otherembodiments are possible.

Input: an email E, a protected organization O

output: a classification corresponding to a conclusion

process: determines a classification of an email received by a protectedorganization

      process detail:          % step 1: fact finding         E.ResetVariables          E.IsFriend := IsFriend(E)         E.IsInternal := IsInternal(E)          E.HowDeceptiveIsSender:= HowDeceptiveIsSender(E,Recipient.contacts)         E.HowManyRecipients:=HowManyRecipients(E,Recipient.address)         E.IsFriend := IsFriend(E,Recipient.contacts)         E.IsInternal := IsInternal(E,Recipient.domain)         E.HasReplyTo:=HasReplyTo(E)          If E.HasReplyTo then           E.IsChameleon:=IsChameleon(E)          If not E.IsChameleonthen            E.HowDeceptiveIsReplyTo:=HowDeceptiveIsReplyTo(E,Recipient.contacts)           E.IsAssociatedReplyTo:=IsAssociatedReplyTo(Recipient.contacts,E)           E.ReplyIsPromiscuous := ReplyIsPromiscuous(E)           E.ReplyToDifferentDomain :=ReplyToDifferentDomain(E)           E.ReplyToPromiscuous:=Promiscuous (E.ReplyTo)          % step2: logic          % logic -- temporary ATO & Spoof detection          If E.HasReplyTo and not E.IsChameleon % a replyto to pay attention to              and            (E.IsFriend or E.IsInternal) % a trustedsender               and            (E.HowDeceptivelsReplyTo >DeceptiveReplyToThreshold) % bad replyto               then           E.Classification := VeryHighRisk          If  E.HasReplyToand not E.IsChameleon % a replyto to pay attention to               and           (E.IsFriend or E.IsInternal) % a trusted sender              and            not E.IsAssociatedReplyTo % sender has notused this before               and            (E.ReplyToDifferentDomainor E.ReplyToPromiscuous)            % the reply-to domain is differentfrom sender domain            % or the sender is promiscuous (in whichcase different            % does not matter)               then           E.Classification := HighRisk            If (E.HowManyRecipients=1) % only one recipient in protected org              and            E.ContentRiskClassification = VeryHighRisk% content bad               then               E.Classification :=VeryHighRisk % upgrade risk          % Here, a message may be sent tothe apparent sender of the message,          % requiring an action inorder for the message to be delivered to its recipients          % logic-- permanent ATO detection       If   not E.HasReplyTo % no replyto              and          (E.IsFriend or E.IsInternal) % a trustedsender               and          (E. HowManyRecipients=1) % exactly onerecipient in protected org               then          E.Classification:= E.ContentRiskClassification % review content          IfE.Classification = VeryHighRisk       then              E.Classification := HighRisk % downgrade to avoid block              % Here, the messages is optionally considered for         % quarantine until the sender has responded to a              % secondary channel challenge, indicating that the message              % should be delivered       % Here, a message may be sentto a valid channel associated with the sender of the message, %requiring an action in order for the message to be delivered to itsrecipients          % logic -- cousin-name detection          if  not(E.IsFriend or E.IsInternal) % not a trusted sender               and           (E.HowDeceptiveIsSender > HighDeceptiveSenderThreshold)           % obviously bad               then           E.Classification := VeryHighRisk          else if not(E.IsFriend or E.IsInternal) % not a trusted sender               and           (E.HowDeceptivelsSender > MediumDeceptiveSenderThreshold)           % likely bad               then            E.Classification:= HighRisk            If E.ContentRiskClassification = VeryHighRisk %risky content               then               E.Classification :=VeryHighRisk % upgrade          else if not (E.IsFriend or E.IsInternal)% not a trusted sender               and           E.ContentRiskClassification = VeryHighRisk % risky content              then               E.Classification := HighRisk %downgrade due to lack of info          % detecting phishing           If not E.Isfriend then               if NewDomain(E.from)then                  {one of the following, based on what we decide on:                    {If PotentialPhishingURLs(E) then                    E.ContentRiskClassification = VeryHighRisk}                    or                     {ProtectPhishingURLs(E)}                 }          % at the very end, when the classificationsof E have been entirely completed            if (E.Classification =Safe) and AddToRecordedReplyTo               then              NowRecordReplyTo(E)            if (E.Classification =VeryHighRisk)               then              Unfriend((E.from).address, C) % remove offender fromfriend list.

ContentRiskClassification

In many contexts, it is important to perform an in-depth scan of theemail contents. In one embodiment, this is performed as follows:

0. Set the content score to zero. In some embodiments, this score isconditionally modified as the following example scan is performed:

1. Does the message have an attachment?

-   -   a. If yes to (1), does the attachment have a high-risk word in        its name?        -   i. If yes to (1a), then add a value to the score for that,            such as 4.    -   b. If yes to (1), was the attachment generated using a free        service?        -   i. If yes to (1b), then add a score for that, such as 7.    -   c. If yes to (1a) or (1b), then scan the contents of the        attachment and add a score related to the result, where this        score may be a value from 0-9, and depend on the presence of        keywords associated with risk. For example, the word “invoice”        may correspond to 2 points, while the word “wire” or the term        “Western Union” may correspond to 4 points.

2. Does the message have a high-risk word in its subject line?

-   -   a. If yes to (2), then add a value to the score for that. For        example, the same scoring as performed in step 1c may be used,        or a similar method with different terms and different scores.

3. Does the message match a vector filter rule?

-   -   a. If yes to (3) then add a value to the score for that, based        on the hit. Vector filter rules are described below.    -   b. Does the vector filter rule correspond to a whitelisted        brand? (In one embodiment, this is determined based on the        family the rule belongs to, where different rules belong to        different families; in another embodiment, the system maintains        a list of whitelisted brands.)        -   i. If yes to (3b) then add a score for that, except if the            sender is associated with the brand (i.e., “Bigfoot” sends            email for JP Morgan, as does JP Morgan.)        -   ii. If yes to (3b) then is the whitelisted brand associated            with URLs?            -   1. If yes, then determine whether the message contains                any URL not associated with the whitelisted brand, and                add a value to the score for that. One example of this                value may be 12.

4. Is there presence of obfuscation in the message (e.g., mixed orhigh-risk charsets)?

-   -   a. If yes to (4), then add a value to the score for that, such        as 9.

5. Is there a likely presence of spam poison? To determine this, acollection of heuristic verifications can be performed. For example, inone embodiment, it is verified whether the message has two text parts,each containing at least 25 characters, and these components areseparated by at least 15 contiguous linefeeds. If this is satisfied,then the message is determined to have a likely spam poison component.

-   -   a. If yes to (5) then add a value to the score for that, such as        7.

6. Does the message match a storyline?

-   -   a. If yes to (6), then add a value to the score for that, such        as a value between 0 and 15, where this value is computed by        matching the content to one or more collections of terms.

Periodic Maintenance:

In addition, in some embodiments, periodical maintenance is performed.Example processes are described:

UpdateFriends

-   -   input: an email address A, contact list C of protected account,        Inbound    -   process: update C, when applicable    -   The variable Inbound is Boolean, and indicates whether the        function is called because as a result of an inbound email with        address A or not (i.e., an outbound email with address A.)    -   For each email E sent to a protected account P, then we call        UpDateFriends(E.from, P.contacts, true)    -   For each email E sent from a protected account P, then we call        UpDateFriends(A, P.contacts, false) for each recipient account A        (i.e., to, cc and bcc)

      detailed process:       If there is a record Ci such that (Ci.A=A)then          If Ci.DateQualified != nil then             IfCi.DateQualified + FriendDelayThreshold < (today′s date)               then             Ci.friend := true          else            If InBound then                Ci.NumberEmailsTo++            else                Ci.NumberEmailsFrom++             IfQualifiesAsFriend (Ci.NumberEmailsTo,Ci.NumberEmailsFrom) then               Ci.DateQualified := (today′s date)          else  %consider creating a record           If not IsInternal(A.domain)            then             Create a new record Ci and insert it in C            Ci.A := A             Ci.DateQualified:=nil            Ci.NumberEmailsTo:=0             Ci.NumberEmailsFrom:=0            Ci.friend := false             If InBound then               Ci.NumberEmailsTo++             else               Ci.NumberEmailsFrom++

-   -   -   In some embodiments, the above function uses the following            internal routine:

QualifiesAsFriend

-   -   input: NumberEmailsTo, NumberEmailsFrom    -   return ((NumberEmailsTo>ToThreshold) or        (NumberEmailsFrom>FromThreshold))    -   Where example values are        -   ToThreshold=2        -   FromThreshold=4

PruneAssociatedReplyTo

-   -   process:    -   For all protected users, review all their contacts Ci.    -   If any Ci has a Ci.AssociatedReplyTo (which is an address) that        is on the global list ChameleonList, then remove that entry        Ci.AssociatedReplyTo. The entry is not harmful, but it is also        not useful.

When: Periodically.

Cousin Clearinghouse

A cousin attack is a type of scam in which a deceptive address is used,whether in an email or in a URL.

Scammers will attempt to trick email receivers by using a close copy ofa legitimate domain. These are called cousin domains. For example,www.pavpal.com looks very similar to www.paypal.com. Scammers can createcousin domains in a variety of ways including adding letters, omittingletters, reversing letters, adding punctuation or using alternatecharacter sets such as Cyrillic to create homographs. Scammers can use adifferent top level domain (TLD) extension such as www.paypal.tv.Scammers can also combine a domain name with other words such aswww.paypal_service.com or create a subdomain such asservice.paypalservice.com. Since the number of possible characterreplacement and word combinations is effectively unbounded, it can bedifficult to predict all possibilities.

In some embodiments, the Cousin Clearinghouse is an enumeration of badcousin domains that email service providers and browsers or browserplugins can query to detect spoofed email and spoofed web page attempts.When mail services or browsers find these domains they can takeappropriate action like block the traffic or provide an in contextwarning to the user. For example:

-   -   The URL www.pavpal.co/login.html appears to be a spoof of the        legitimate site PayPal.com. This is likely a scam site and you        should proceed. Click Here to be Safe, but if you wish to        proceed please confirm. YES, I UNDERSTAND THE RISK.    -   The email below has been sent by a sender that appears to have a        spoofed domain. This is likely a scam, so you should delete this        email.    -   The email below has a contains a link that appears to be a        spoofed domain. This is likely a scam, so you should not        proceed.    -   You have a received and email from a sender that appears to come        from a spoofed domain. This email has been quarantined for        security, but can be viewed if you wish. Please confirm if you        want to view this email: YES, I UNDERSTAND THE RISK.

In some embodiments, the Cousin Clearinghouse can be queried via asecure internet connection or a cached list of bad domains can be pushed(or pulled) from a service.

In some embodiments, the Cousin Clearinghouse can be populated by aconstant proactive scanning of new domain registrations. As new domainsare published they can be detected and scored based on a variety offactors including:

-   -   Contains one or more words or names representing a known brand.        For example, contains “paypal”.    -   Contains one or more words or names similar to a known brand.        For example, contains “pavpal”    -   Contains one or more homographs that makes the domain appear        similar to a known good domain. For example, www.paypal.com        could be represented with Cyrillic ‘a’.    -   Is hosted by a service provider that has been previously        detected to have hosted cousin domains.    -   Is hosted by a service provider that is not in the country where        the domain is pretending to be. For example, www.pavpal.com        could be hosted in the Ukraine with an all English web site.    -   Cousin domain contains text or image content that is the same or        similar to the known good site.    -   Uses a different TLD than the known good site like www.paypal.co    -   A person can manually review the email

In some embodiments, the Cousin Clearinghouse can also receive reportsfrom corporations or end users that find suspect domains names. Thesecan be reported in a variety of ways, including:

-   -   Suspicious emails can be manually forwarded by users to an        automated email address like spoof@zapfraud.com.    -   Emails can be automatically forwarded if they are found to be        suspicious by filters at the mail service.    -   Email addresses can be manually reported through a web page    -   A list of email addresses can be uploaded as a file through a        web page    -   Email addresses can be automatically reported from a mail        service via an internet service connection

In some embodiments, the domains found in these reports can be validatedagainst the criteria identified above.

Detecting Relationships Through Display Name Management

In some embodiments, Display Names can be included before the actualemail address. For example, “Super Genius” is the Display Name in “SuperGenius” <wiley.e.coyote@acme.com>. Outbound email usually contains aDisplay Name as part of the From address.

Receiving mail services or mail clients often capture this Display Nameso that it can be used in mail clients when the receiver wants to send amessage back later, since it is much easier to show a display name suchas “Bob Smith” rather than a more convoluted email that it represents,such as smith-b181703@obscuredomain.com. The previously received DisplayName is then automatically used in the To: field of outbound email tothe known account. So a sender that knows the receiver should use thecorrect Display Name when sending to that email. In one example, if theDisplay Name is something such as “Happy User” instead of “Bob Smith”this is a strong indication that the sender probably does not know thereceiver. If the Display Name is “Robert Smith” when Bob has never usedthat Display Name, then this is a strong indication that this is someonethat Bob does not know. If the Display Name is missing completely, thatmay also be an indication that the sender does not know the receiver. Ifthe sender does not include the proper Display Name for the receiver,the message can be scored as being more suspicious. This can be used byitself or in combination with other scam indicators to decide thedisposition of the message.

In some embodiments, display names are modified to make them harder toguess by senders that do not really know the receiver. For example, “*Bob Smith *” or “* Bob ** Smith ***” or similar variations would not beeasy to guess by scammers. In some embodiments, changes are randomizedper account so that they cannot be easily guessed by a scammer. If amessage contains, for example, a To: address with a plain “Bob Smith,”in some embodiments, it is scored as a potential scam since it lacks theadditional decoration that distinguishes display names that originatedfrom the account holder.

In some embodiments, the display name is automatically modified tochange based on a schedule and/or when an event occurs. For example, inJanuary the display name for the account could be “* Bob Smith *”, thenchanged to “! Bob Smith !” in February and “** Bob Smith !” in March.Alternatively, the change can be triggered when too much SPAM email isreceived by an account. By switching to a new display name, olderdisplay names can be recognized as potentially suspicious.

In various embodiments, the Display Names can include Unicode charactersfor example “⋆ Bob Smith ★” or can use homograph characters such as aCyrillic ‘h’ in “Bob Smith’ or invisible characters such as Tab or otherinvisible Unicode characters

Another example approach is to use a title such as “Bob Smith, CFO” or“Bob Smith C.F.O.” in the display name so that only senders that havereceived email from Bob would know what Bob appends.

In some embodiments, by changing the Display Name and recording when itwas changed, it is possible to recognize/determine how old a connectionis to a previous sender.

Where Display Names can be accessed in a central location, in someembodiments, the modification of Display Names can be modifiedprogrammatically or manually. For example, if Linux display names arestored in the /etc/passwd file such as:

bob:x:1001:1001:*Bob Smith*:/home/bob:/bin/bash

these can be easily accessed for updates. In other cases, the displaynames may be stored in a database, such as a database containingMicrosoft Exchange Server accounts, or directory structure like LDAP.

Additional Figures

FIG. 3 illustrates an example process to determine that an account is afriend. In some embodiments, the example process 300 of FIG. 3 isexecuted using the scam detection system described herein (e.g., scamdetection system 100). At 301, incoming email is accessed, and thecontents of the “from” field are determined. In an alternativeembodiment, the contents of the “sender” field are also determined. Thisneed not be done in real-time, but can be done in batch mode, includingat account enrollment or subsequent processing of (all) email messageheaders.

At 302, the system accesses an outgoing or sent emails, and determinesthe contents of the “to”, “cc” and “bcc” fields. This need not be donein real-time, but can be done in batch mode, including at accountenrollment. At 303, one or more counters are conditionally increased,based, for example, on the accounts determined at steps 301 and/or 302.For example, if at 301 it is determined that an email address E1 is usedfor sending an email to a protected account, then at step 303, a counterassociated with E1 and with incoming email is increased. Similarly, if,for example, at 302 it is determined that an email address E2 is arecipient of an email from a protected account, then at step 303, acounter associated with E2 and with outgoing email is increased. Thus,in one embodiment, there is one counter for each email address fromwhich email is received or to which email is sent. In one embodiment,the increase of the counter(s) is conditional on a maximum value for thecounter(s) not having been reached. At 304, the one or more countersassociated with an email account (E1 or E2) identified at 301 and/or 302are compared to one or more thresholds. At 305, it is determined whetherthe one or more counters meet or exceed one or more thresholds. In oneembodiment, all counters associated with an email account have to exceedtheir respective thresholds, whereas in another, at least one of thecounters associated with an email account has to exceed its associatedthreshold. The email account is E3, which may be different from E1 andE2, or which may match one or both of these. If the threshold wasexceeded then step 306 is performed; otherwise, step 307 is performed.At step 306, a time stamp is recorded. This corresponds to when theemail account was determined to have met the requirement for being afriend, based, for example, on at least one of incoming traffic andoutgoing traffic, or a combination of these. At 307, at least onetimestamp is reviewed to determine if it is sufficiently old, i.e., asufficient time has elapsed since the timestamp was recorded. In oneexample embodiment, that time is two weeks or anything exceeding twoweeks. At 308, the comparison is performed if the timestamp is oldenough, and if it is, step 309 is performed; otherwise step 310 isperformed. At step 309, it is recorded that the account E3 for which thetime-stamp was found to be old enough is a friend of the protectedaccount. An example embodiment is described above, in the procedurecalled “UpdateFriends”. In an alternative embodiment, the determinationof who is a friend is not done with respect to a protected account, butinstead, with respect to a protected organization. That would mean thatthe counters described above would not be specific to a unique protectedaccount within an organization, but instead, all users within the sameorganization would use the same counters. In other words, if one emailto a first user in an organization is received, and then a second emailto a second email in the same organization is received, and the emailsare from the same sender, then the same counter would be increasedtwice.

FIG. 4 illustrates an example process to determine that an email senderis trusted. In some embodiments, the example process 400 of FIG. 4 isexecuted using the scam detection system described herein (e.g., scamdetection system 100). It is determined whether the party is internal atstep 401. In some embodiments, two parties are internal to each other ifthey have email addresses within the same organization, and this is anorganization that is being protected. It is not necessary for them tohave the same domain name in their email addresses, as someorganizations may use multiple domains. In some embodiments, a list ofassociated domains is consulted to determine whether a party isinternal. In addition, an enterprise can add—temporarily orpermanently—domains or email addresses corresponding to collaborators,and to personal email addresses of employees of the organization orcollaborating organizations. If a party matches such a list, in someembodiments, it is considered internal. If a party is internal, then theprocessing proceeds to 404. If not, it is determined at step 402 whetherthe party under consideration is a friend. In some embodiments, a partyis a friend of a protected account if this has been recorded, forexample, at step 309 in the example process described in conjunctionwith FIG. 3 . As is also described in the exemplary embodiment, in someembodiments, a party is a friend if it belongs to an organization thatis a friend of the party relative to which the determination is made. Ifthe party being considered is a friend, then step 404 is performed,otherwise step 403. At step 403, a transitive closure algorithm isevaluated based on a configuration associated with the protectedaccount. In one embodiment, the transitive closure algorithm specifiesthat any friend of a party who is internal is a friend. Alternativetransitive closure algorithms can be used. If the party considered is inthe transitive closure, the processing continues to step 404, otherwiseto step 405. At step 405, the processing to determine that an emailsender is trusted concludes. At step 404, the party is set to betrusted.

FIG. 5 illustrates an embodiment of a simplified non-monotonicallyincreasing filter. In some embodiments, the example logic of FIG. 5 isimplemented using the scam detection system described herein (e.g., scamdetection system 100). At 501, an incoming email is scanned. At 502, itis determined whether there are signs of an account-takeover (ATO) inthe incoming email. In one embodiment, this test also includes adetermination of likely spoofing attempts. Examples of such signsinclude new signature files, new display names, high-risk email content,and email delivery paths that are abnormal, including containing atleast two more hops than typically recorded for this sender or includingnodes that are not normally on the delivery route for this sender. Ifthere are signs of ATO, then the logic/process proceeds to 506,otherwise to 503. At 503, it is determined if the email has a reply-toaddress that is not previously associated with the sender for emailsthat were considered safe. If this is determined to be true, then theprocess proceeds to step 506, otherwise to 504. In step 504, it isdetermined whether the sender email address is deceptive. In someembodiments, this corresponds to the sender obtaining a deceptive scoreexceeding a threshold, such as at least 70 out of 100, where an exampledeceptive scoring algorithm is described in the pseudo code of theexemplary embodiment. If the deceptive score exceeds the threshold, thenthe address is considered deceptive, and the process continues to step508, otherwise to step 505. In 505, the non-monotonic scan concludes.Step 506 and step 508 both determine whether the sender is trusted, andthe same processing can be used to determine this. In some embodiments,they are different in terms of the action, though. At step 506, an emailis considered dangerous if the sender is trusted, whereas at step 508,an email is considered dangerous if the sender is not trusted. This isan example of a non-monotonic combining logic. If the email isconsidered dangerous, then the process continues to 507, where it islabelled high-risk; otherwise to 505.

FIG. 6 illustrates an alternative embodiment of a non-monotoniccombining logic. In some embodiments, the example logic of FIG. 6 isimplemented using the scam detection system described herein (e.g., scamdetection system 100). At 601, an incoming email is scanned, similarlyto at 501 of FIG. 5 . In 602, it is determined whether the sender istrusted, similar to at 506 and 508 of FIG. 5 . If the sender isdetermined to be trusted, then the logic/process continues to 603,otherwise to 605. At 603, it is determined whether the email has signsof account take-over, using a process that could be essentially the sameas in 502. If it is, then the process continues to 606, otherwise to604. At step 604, it is determined if the email has a new reply-toaddress, similarly to as was also described at 503 in FIG. 5 . If thisis determined to be true, then the process continues to 606, otherwiseto 607. At step 605, it is determined whether the sender address isdeceptive, which can be done, for example, similarly to as at 504 ofFIG. 5 . If that is true, then the process continues to 606, otherwiseto 607.

FIG. 7 illustrates a second alternative embodiment of a non-monotoniccombining logic. In some embodiments, the example logic of FIG. 7 isimplemented using the scam detection system described herein (e.g., scamdetection system 100). At 701, an incoming email is scanned, forexample, similarly to as at 501 of FIG. 5 . At 702, it is determinedwhether the sender is trusted, for example, similarly to as described at602 of FIG. 6 . If the sender is trusted, the logic/process continues to703, otherwise to 704. At 703, it is determined if the email has a newreply-to address, for example, similarly to as at 503 of FIG. 5 . If itdid, then the logic/process continues to 706, otherwise to 705. At 704,it is determined whether the sender email address is deceptive,similarly to as at 605. If it is determined to be deceptive, then thelogic proceeds to 709, otherwise to 710. At 705 and 710, it isdetermined whether the email has high-risk content. In some embodiments,this is done by scanning the text portions for keywords such as“invoice” and “wire”, that are associated with high risk, and to convertpotential attachment to text and determine whether this text containskeywords associated with high risk. Steps 705 and 710 can be the sameprocess, except that in some embodiments, the determinations result indifferent actions. At 705, high risk content causes step 708 to beperformed, where the incoming email is marked up, whereas at 710, highrisk content causes step 709 to be performed, where the email isblocked. A negative determination in both 705 and 710 leads to going tostep 707, where the processing of the email ends. Step 708 correspondsto an action taken on emails that are high risk but which the recipientwould not want to lose if the emails are legitimate. In someembodiments, these emails are marked up with warnings. Alternatively,these emails are quarantined. Step 709 corresponds to high-risk emailsthat are blocked. In this example, it can be seen that the actions takenafter a determination in steps 705 and 710 depends on the determinationmade in step 702, which is a non-monotonic filtering logic.

FIG. 8 illustrates an example process for classification of primaryrisks associated with an email, using a non-monotonically increasingcombining component. In some embodiments, the example process 800 ofFIG. 8 is executed using the scam detection system described herein(e.g., scam detection system 100). At 801, an email is scanned. At 802,it is determined whether the sender is trusted. If the sender istrusted, the process proceeds to 803, otherwise 804. At 803, it isdetermined if the email has a new reply-to address. If it does, then theprocess proceeds to 805, otherwise 806. At 805, it is determined whetherthe email has a delivery path that is anomalous, such as containingnodes that have not previously been recorded as being on the path foremails associated with the sender of the scanned email, or having a paththat is at least, for example, two hops longer than previous deliverypaths associated with the sender of the email. If the delivery path isanomalous, then the process proceeds to 809, otherwise 808. At 806 and807, it is determined if the email has high-risk content. If this istrue in the determination at 806, then the process continues to 801,otherwise to 811. If the email is found to have high-risk content at807, then the process continues to 813, otherwise 811. Here, 808corresponds to temporary account take-overs, which are characterized byan attacker with temporary access to an account he has gainedillegitimate access to (e.g., by phishing the owner), while the owner islikely to also have access. 809 corresponds to spoofed emails, such asemails sent through open relays. 810 corresponds to permanent accounttake-overs, which are characterized by an attacker with access to anaccount he has gained illegitimate access to (e.g., by phishing theowner), while the owner is likely not to have access to the accountanymore. This means that it may not be meaningful to alert the accountowner by sending a message to the email account. 811 corresponds to anemail that is likely to be safe. Step 812 corresponds to a so-calledcousin-name attack. This is an attack in which a criminal creates anaccount or registers a domain with the intent to deceive a recipient tobelieve that he is somebody they trust. 813 corresponds to a high fraudrisk that is not classified. Additional tests and processing can beapplied to emails that result in this determination, to further identifywhat attack they are associated with. It may be useful to automaticallydetermine the likely cause of a problem, as this helps remediate theproblem.

For example, if it is determined that an account is likely to havesuffered a temporary account take-over (808), then an alert is sent tothe administrator of the account, who may turn off any remote access tothe account while still allowing access from the enterprise site. Alltraffic from the account is then be automatically marked up withwarnings by the system that first detected the problem, until theadministrator responds that the password has been reset. This includesemail traffic to other users than the user whose received email set offthe alert. If a permanent account take-over is suspected, on the otherhand, an alert is sent to the administrator, who then may investigatewhether this is correct, since the false positive rates of thisdetermination are substantially higher than for temporary accounttake-overs. If the administrator determines that the account was indeedtaken over, he may lock the offending account down. Until a confirmationis received from the admin, the system detecting the possible permanentaccount take-over, in some embodiments, places all emails from theaccount identified as suffering a permanent account take-over inquarantine if they contain an attachment, and mark them up with warningsotherwise. Compare this reaction to that in 812, where traffic from thesender of the scanned email would simply be blacklisted, and all emailfrom this sender rejected onwards, whether sent to the same recipient asthe scanned email, or to another recipient protected by the system.

FIG. 9 illustrates an example embodiment of a process to identify whatmessages should be quarantined based on both high risk and a reasonablelikelihood of being legitimate. In some embodiments, the example process900 of FIG. 9 is executed using the scam detection system describedherein (e.g., scam detection system 100). At 901, it is determinedwhether an email is considered high risk based, for example, oncontents, headers, attachments and transmission context, and history. Insome embodiments, the transmission context includes other messages inthe same thread, and the history includes past classifications ofmessages from the same sender. If the email is considered high risk,then the process proceeds to 902, otherwise to 903. At both 902 and 903,it is determined whether the message comes from a trusted party. At 902,if it does, then the process proceeds to 904, otherwise to 905. At 903,if it does, then the process continues to 906, otherwise 907. Here, inthis example, 904 corresponds to quarantining or marking the message up.In some embodiments, the decision of whether to quarantine or mark up isbased on additional determinations and preferences set by the user or anadmin associated with the user; where the user corresponds to the partywho is the recipient of the email. At 905, the email is blocked. In someembodiments, blocking also includes reporting of the message to anadmin, whether individually or in aggregate. An aggregate reporting cannotify the admin of how many messages sent from or to a particular userwere blocked, or what fraction of such messages was blocked. At 906, themessage gets priority delivery to the recipient. In some embodiments,that is the same as delivery, but in other embodiments, it includes anotification by SMS, a highlighting of the email, a reordering of theemail inbox to highlight the email, or any other appropriate mechanism.At 907, the email is delivered.

FIG. 10 illustrates an embodiment of a quarantine process using asecondary channel for release of quarantined messages. In someembodiments, the example process 1000 of FIG. 10 is executed using thescam detection system described herein (e.g., scam detection system100). At 1001, an email message is quarantined. At 1002, the sender ofthe quarantined message is notified using a secondary channel, such asSMS. At 1003, the system awaits a response to the notification. If thereis a time-out, i.e., there is no response before a threshold time haselapsed, where this threshold time, as one example, is 5 days, then theprocess continues to 1004; if there is a response, then the processcontinues to 1005. At 1005, it is determined whether the response isvalid. For example, a valid response to the notification can include aPIN, the word “yes” or an empty response, and an invalid responseanother message. If the response is not valid, then the process proceedsto 1003; otherwise to 1006. At 1004, the quarantined message is removedfrom quarantine and blocked. At 1006, the quarantined message is removedfrom quarantine and delivered to the recipient. In addition, a messagecan be delivered to the recipient by an action by the recipient, as willbe described below in conjunction with FIG. 11 .

FIG. 11 illustrates an example embodiment of a process for processing ofa quarantined email message. In some embodiments, the example process1100 of FIG. 11 is executed using the scam detection system describedherein (e.g., scam detection system 100). At 1101, the email isquarantined. At 1102, the recipient is notified that she has aquarantined message. This notification can include information about oneor more quarantined messages, including the sender, portions of themessage, and information about why the message was quarantined. Forexample, the notification can specify that there are two quarantinedmessages; when they arrived; the email addresses and display names ofthe senders; the subject lines of the messages; and alerts specifyingthat the first message was sent from a sender with a name similar to atrusted sender, and the second message has signs of having been sentfrom a hacked email account. At 1103, the system receives an actionrequest from a user. If this is “deliver” then the process continues to1104, where the message is removed from quarantine and delivered. If theaction request is “block” then the message is removed from quarantineand blocked. If the action request is “blacklist” then the sender isblacklisted. Note that if a message is removed from the quarantine inthe process illustrated in FIG. 11 , then it does not matter whether thesender responds with a valid response at 1005 of FIG. 10 —since themessage has been removed, it can no longer be delivered at 1006 of FIG.10 .

FIG. 12 illustrates an example of the three stages in one embodiment ofa 2FA confirmation process. In some embodiments, the example process ofFIG. 12 is executed using the scam detection system described herein(e.g., scam detection system 100). Stage 1 corresponds to unit 1200,stage 2 to unit 1210, and stage 3 to unit 1220. At 1201, the systemidentifies a trusted user and goes to 1202. At 1202, it is determinedwhether the trusted user has validated channel data associated with hisor her record kept by the system. If this is true, then the processcontinues to 1203, otherwise to 1204. At 1204, the system generates arequest for channel data. In one embodiment, this request is sent to thetrusted user by email. In another embodiment, the channel data isrequested from a user associated with an account receiving at least oneemail from the trusted user. In yet another embodiment, storagemaintained by the system or its users is searched to identify channeldata associated with the trusted user. At 1205, channel data is receivedin response to the request, and at 1206 it is verified whether thechannel data is valid. In one embodiment, this involves sending an SMSto the trusted user using the newly received channel data as a recipientaddress for the SMS, where the SMS contains a code that is generated bythe system. Further, an email can be sent to the trusted user,containing a hyperlink. When the user clicks on the hyperlink, he or shecomes to a webpage controlled by the system, in which the code sent bySMS can be input. If this is correctly input, then the channel data isconsidered valid. In another embodiment, a user associated with theprotected organization is asked to confirm that the contact informationis valid. If this is done, then the channel data is considered valid. Ifthe channel data is considered valid in 1206, then the process continuesto 1207, otherwise to 1203. At 1203, the attempt to register channeldata ends for now. At 1207, the validated channel data is added to arecord associated with the trusted user.

1210 corresponds to stage 2. At 1211, the system scans an incoming emailand proceeds to 1212. At 1212, it is determined whether the email ishigh risk. If that is true, then the process continues to 1213,otherwise to 1224. At 1213, it is determined whether the email is from atrusted sender. If yes, then the process proceeds to 1214, otherwise to1215. At 1215, the email is blocked. At 1214, the email is quarantined,after which the system proceeds to 1216, where a 2FA confirmationrequest is attempted to be generated. An example request is “Did yousend an email to Alice with subject ‘Here is my invoice’? If so, respondY to this SMS. To report abuse, respond N.” If there is valid channeldata associated with the sender of the email, then the 2FA confirmationrequest is generated and sent to the valid channel address, after whichthe system proceeds to 1221. If there is no valid channel dataassociated with the sender, then a registration request similar to thatat 1204 is generated and transmitted. After it has been received anddetermined valid, the email is marked up and moved to the inbox (notshown in the figure).

1220 corresponds to stage 3. At 1221, the system received a response tothe 2FA confirmation request; this response is referred to as theconfirmation. At 1222, it is determined whether the confirmation isvalid. For example, assume the request sent at 1216 is “Did you send anemail to Alice with subject ‘Here is my invoice’? If so, respond Y tothis SMS. To report abuse, respond N.” In this case, the response “Y” isconsidered a valid response. If the response is valid, then the processcontinues to 1224, where the email is moved from quarantine to therecipient inbox(es). If the responds is not valid, then at 1223, anoptional notification is sent to the apparent sender of the email. Inaddition, the system can flag the email as abusive, and this flaggedemail can be forwarded to an admin, or used for system trainingpurposes, or both. In some embodiments, the 2FA confirmation process isused to reduce the impact of spoofed BEC emails, and emails that aresent from legitimate accounts of trusted people, where these accountshave been taken over by scammers, e.g., using phishing attacks.

In an alternative embodiment, 2FA tokens are used instead of registeredchannels. In that context, stage 1 is not needed, and is replaced by thedistribution of the 2FA tokens. Furthermore, at 1216, a message is sentto the email of the sender, asking, for example, the sender to click ona link and enter the code from the 2FA token. That value is received at1221, and validated at 1222. In some embodiments, a valid response isone that matches the token output. If the response is not valid, thesender is notified to try again at 1223, after which the system getsready to receive a new confirmation at 1221.

FIG. 13 illustrates an example embodiment of processing associated withsending a request to an account associated with the apparent sender ofan email. In some embodiments, the example process 1300 of FIG. 13 isexecuted using the scam detection system described herein (e.g., scamdetection system 100). At 1301, the prevalent risk of the email isdetermined. At 1302, it is determined whether the prevalent risk isaccount take-over. If it is, then the process continues to 1304,otherwise to 1303. At 1303, it is determined whether the prevalent riskis spoofing. If it is, then the process proceeds to 1306, otherwise1305. At 1305, a filtering action is performed based on the identifiedprevalent risk and the severity of this risk. Example actions includeblocking the message, placing the message in quarantine, marking themessage up, and notifying an admin of the message, or combinations ofsuch actions. At 1306, a request is sent to the apparent sender of themessage. Example requests are shown in FIGS. 14 and 15 . At 1304, it isdetermined whether there is a valid channel associated with the sender.If there is, then the process continues to 1307, otherwise to 1306. At1307, a request is sent to an address that is a valid channel associatedwith the apparent sender of the message. An example request is shown inFIG. 15 . At 1308, the system verifies the response to the request,whether this was sent at 1306 or 1307. If the response is determined tobe valid at 1309, then the process proceeds to 1311, otherwise 1310. At1310, the message is not delivered, but is preferably blocked. At 1311,the message is delivered. In one embodiment, all or some of all blockedmessages are sent to an admin or a security agent for furtherprocessing. The decision of whether to forward blocked messages to anadmin, to a security agent, to both or neither depends on theconfiguration and on the flow in which the message was determined to behigh-risk, and consequently blocked.

FIG. 14 illustrates an example embodiment of a request. In someembodiments, this type of request is sent to the apparent sender of anemail that is determined to be at high risk of being spoofed. At 1401,such a request is shown. At 1401, the recipient of the request is askedto click on a hyperlink 1402 to have the email delivered. In analternative embodiment, the recipient of the request is asked to replyto the request to have the email delivered. If the recipient of therequest clicks on 1402 then a webpage 1410 is shown to him or her. Inthis, the person is asked to enter a secondary email address at 1411 anda phone number at 1412. These are referred to herein as channeladdresses. After receiving channel addresses, the system initiates avalidation attempt. In some embodiments, this involves sending a messageto each of the received channel addresses, asking the recipient to clickon a hyperlink or respond in order to have the channel addressvalidated. In some embodiments, the hyperlink is unique, allowing thesystem to determine the account associated with the click. Similarly, aresponse to the request by clicking “reply” allows the system toidentify who responded.

FIG. 15 illustrates an example embodiment of a request that is avariation of the request in FIG. 14 . The request at 1501 is sent inresponse to an email being determined to be at high risk to beassociated with spoofing or with an account take-over. If the recipientof the request 1501 clicks on the hyperlink 1502 or replies to therequest 1501 with a message containing the word “yes” then the emailassociated with high risk is delivered to its recipient.

FIG. 16 illustrates an example embodiment of a cousin clearinghouse. Insome embodiments, the example cousin clearinghouse shown here is aportion of the scam detection system described herein (e.g., analternate view of scam detection system 100 of FIG. 1 ). In the exampleof FIG. 16 , the Cousin Clearinghouse (1606) polls Domain Registries(1607) to identify domains that may be close copies of Known Good (1608)domains and scores them. Domains that exceed a scam threshold are addedto the Bad Domains (1609) repository. The Cousin Clearinghouse securelyrefreshes Cached Bad Domains (1604) list at a mail service providerthrough an Updater (1610) that resides at the mail service provider. Themail service (1603) reads the Cached Bad Domains (1604) and appliesdomain filters to the incoming or outgoing mail stream. Users (1601) cansafely access the Internet through a web browser (1605) that checks URLsagainst the Cousin Clearinghouse (1606) and blocks access to baddomains. Users (1601) read email from a mail server (1603) through amail reader (1602). If a user (1601) finds a suspect domain they canreport it to the Cousin Clearinghouse (1606) via a communicationschannel such as email. In some embodiments, a Mail Service (1603) canautomatically send suspect domains to the Cousin Clearinghouse when theyare found by other filters.

In one embodiment, the browser maintains a cache of bad domains toreduce the traffic to the Cousin Clearinghouse.

In one embodiment, a score is maintained for each Bad Domain. In someembodiments, smart filters at the mail server or the browser can decideappropriate actions based on this score. For example, additionalinformation such as suspicious email body content or the user's browsersecurity settings can be used to determine whether the content isblocked or a warning is shown.

In one embodiment the known good list entries with scores can also beprovided in addition or instead of the bad list. This allows refineddecision making by the mail server or browser. For example, if a domainis known to be good rather than unknown, the content is less likely tobe scam or even spam.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A system for detection of email risk, comprising:a processor configured to: automatically determine that a first party isconsidered by the system to be trusted by a second party, based on atleast one of determining that the first party is on a whitelist and thatthe first party is in an address book associated with the second party;receive a message addressed to the second party from a third party;perform a risk determination of the message by determining whether themessage comprises a hyperlink and by determining whether a display nameof the first party and a display name of third party are the same orthat a domain name of the first party and a domain name of the thirdparty are similar, wherein similarity is determined based on having astring distance below a first threshold or being conceptually similarbased on a list of conceptually similar character strings; responsive todetermining that the message poses a risk, automatically perform asecurity action comprising at least one of marking the message up with awarning, quarantining the message, performing a report generating actioncomprising including information about the message in a reportaccessible to an admin of the system, and replacing the hyperlink in themessage with a proxy hyperlink; and a memory coupled to the processorand configured to provide the processor with instructions.
 2. The systemof claim 1 wherein a request associated with the proxy hyperlink causesthe system to: determine the hyperlink from the proxy hyperlink;determine whether a site associated with the hyperlink is associatedwith risk; and based on the determination whether the site associatedwith the hyperlink is associated with risk, cause a warning to bedisplayed or a redirection to be made from the proxy hyperlink to thehyperlink.
 3. The system of claim 2 further comprising determiningwhether the site associated with the hyperlink is associated with riskbefore the request associated with the proxy hyperlink is received. 4.The system of claim 2 further comprising determining whether the siteassociated with the hyperlink is associated with risk in response toreceiving the request associated with the proxy hyperlink.
 5. The systemof claim 1 wherein in response to receiving a request associated withthe proxy hyperlink: determine the hyperlink from the proxy hyperlink;verify content of a site associated with the hyperlink, and based on aresult of the verification, cause at least one of a warning to bedisplayed and a redirection to be made from the proxy hyperlink to thehyperlink.
 6. The system of claim 1 wherein the proxy hyperlink encodesat least a portion of the hyperlink.
 7. The system of claim 1 whereinthe security action comprises at least one of: initiating a multi-factorauthentication verification, modifying the display name of the message,transmitting a notification or a warning to an address associated withthe second party, collecting information comprising at least one of anIP address, a cookie, and browser version information, and transmittinga confirmation request to an address associated with the first party,the confirmation request comprising at least a portion of the message.8. The system of claim 7 wherein a confirmation received in response tothe confirmation request comprises at least one of an entered code or aclicked link, wherein the link is included in the confirmation request.9. The system of claim 8 wherein information associated with the clickedlink is collected, wherein the information comprises at least one of theIP address, the cookie, and the browser version information.
 10. Thesystem of claim 1 wherein the risk determination is further based atleast in part on at least one of: an indication of spoofing, anindication of account takeover, a presence of a reply-to address, ageographic inconsistency, detection of a new signature file, detectionof a new display name, detection of high-risk email content, detectionof an abnormal delivery path, and based on analysis of attachments. 11.The system of claim 1 wherein an address associated with the first partyis determined to be a secondary communication channel associated with atleast one of the first party and an admin associated with the firstparty.
 12. The system of claim 1 wherein the security action furthercomprises transmitting a confirmation request to an address associatedwith the first party, the confirmation request comprising at least aportion of the message, wherein the message is delivered to the secondparty based on verification of information received in response to theconfirmation request.
 13. A system for determining whether an electronicmessage is deceptive, comprising: a processor configured to:automatically determine whether a first party is considered trusted by asecond party, based on at least on one of determining that the firstparty is on a whitelist and that the first party is in an address bookassociated with the second party; receive a message addressed to thesecond party from a third party; determine if the received message posesa risk by determining that a display name of the first party and adisplay name of third party are the same or that a domain name of thefirst party and a domain name of the third party are similar, whereinsimilarity is determined based on having a string distance below a firstthreshold or being conceptually similar based on a list of conceptuallysimilar character strings; responsive to the first party is consideredtrusted by the second party, and the received message is determined topose a risk, determine that the message is deceptive; responsive to adetermination that the first party is not considered trusted by thesecond party, determine that the message is not deceptive; responsive tothe message being found deceptive, automatically perform a securityaction and a report generation action without having received any userinput from a user associated with the second party in response to themessage, wherein the security action comprises marking the message upwith a warning or quarantining the message, wherein the reportgenerating action comprises including information about the receivedmessage in a report accessible to an admin of the system; and responsiveto the message being found not deceptive, deliver the message to thesecond party; and a memory coupled to the processor and configured toprovide the processor with instructions.
 14. A method for detection ofemail risk, comprising: automatically determining that a first party isconsidered trusted by a second party, based on at least one ofdetermining that the first party is on a whitelist and that the firstparty is in an address book associated with the second party; receivinga message addressed to the second party from a third party; performing arisk determination of the message by determining whether the messagecomprises a hyperlink and by determining whether a display name of thefirst party and a display name of the third party are the same or that adomain name of the first party and a domain name of the third party aresimilar, wherein similarity is determined based on having a stringdistance below a first threshold or being conceptually similar based ona list of conceptually similar character strings; responsive todetermining that the message poses a risk automatically performing asecurity action comprising at least one of marking the message up with awarning, quarantining the message, performing a report generating actioncomprising including information about the message in a reportaccessible to an admin, and replacing the hyperlink in the message witha proxy hyperlink.
 15. The method of claim 14 wherein a requestassociated with the proxy hyperlink results in the method: determiningthe hyperlink from the proxy hyperlink; determining whether a siteassociated with the hyperlink is associated with risk; and based on thedetermination that the site associated with the hyperlink is associatedwith risk, causing a warning to be displayed or a redirection to be madefrom the proxy hyperlink to the hyperlink.
 16. The method of claim 15further comprising determining whether the site associated with thehyperlink is associated with risk before the request associated with theproxy hyperlink is received.
 17. The method of claim 15 furthercomprising determining whether the site associated with the hyperlink isassociated with risk in response to receiving the request associatedwith the proxy hyperlink.
 18. The method of claim 14 further comprisingin response to receiving a request associated with the proxy hyperlinkdetermining the hyperlink from the proxy hyperlink; verifying content ofa site associated with the hyperlink, and based on a result of theverification, causing a warning to be displayed or a redirection to bemade from the proxy hyperlink to the hyperlink.
 19. The method of claim14 further comprising causing the proxy hyperlink to encode at least aportion of the hyperlink.
 20. A method for determining whether anelectronic message is deceptive, comprising: automatically determiningwhether a first party is trusted by a second party, based on at least onone of determining that the first party is on a whitelist and that thefirst party is in an address book associated with the second party;receiving a message addressed from a third party distinct from the firstparty and addressed to the second party; performing a risk determinationof the received message to determine if the received message poses arisk by determining that a display name of the first party and a displayname of third party are the same or that a domain name of the firstparty and a domain name of the third party are similar, whereinsimilarity is determined based on having a string distance below a firstthreshold, or being conceptually similar based on a list of conceptuallysimilar character strings; responsive to the first party being trustedby the second party and the received message is determined to pose arisk, determining that the message is deceptive; responsive to adetermination that the first party is not trusted by the second party,determining that the message is not deceptive; responsive to the messagebeing found deceptive, automatically performing a security action and areport generation action without having received any user input from auser associated with the second party in response to the message,wherein the security action comprises marking the message up with awarning or quarantining the message, wherein the report generatingaction comprises including information about the received message in areport accessible to an admin of the system; and responsive to themessage being found not deceptive, delivering the message to the secondparty.